A RAG knowledge assistant with LangGraph and pgvector

The scenario

Imagine a product team with setup guides, release notes, and internal support documentation in separate repositories. Engineers repeat the same searches, while an unrestricted chatbot could expose material to the wrong person. The product goal would be a small knowledge assistant that can explain an answer and show exactly which permitted documents informed it.

Start with a document lifecycle

An ingestion worker would read approved sources, split documents at meaningful headings, and attach a source URL, revision, owner, and access group to each chunk. A content hash would let unchanged chunks retain their embeddings. Removed documents would need a deletion path, rather than remaining searchable indefinitely.

PostgreSQL would hold both the document metadata and the vectors through pgvector. That keeps ownership and retrieval data close together. The embedding model and chunking strategy would be versioned so that a later migration can rebuild the index without mixing incompatible vectors.

Keep retrieval separate from generation

FastAPI would authenticate the request and derive document permissions from the application session. The retrieval query would enforce that scope before any text reaches a language model. A client-supplied team ID would never be enough to authorize access.

LangGraph could express the workflow as explicit steps: retrieve context, check evidence, generate an answer, and format citations. Retrieved text would be treated as source material, not as instructions to call tools or change permissions. For a question with weak evidence, the assistant would return the available sources and ask for clarification.

Evaluate answers against a small, maintained dataset

A useful first evaluation set would include known-answer questions, questions with no documented answer, and questions about documents the test user cannot access. Reviewers would score whether the cited passage actually supports the answer. A fluent response alone would not count as a pass.

The first release could use a simple retrieval pipeline before adding agent loops. Log document identifiers, workflow version, timing, and errors with a defined retention period; avoid copying confidential prompts into unrestricted logs.

What to validate

• Deleting or changing a source updates retrieval results.
• A user cannot retrieve another access group’s documents.
• Unsupported questions produce an explicit uncertainty response.
• Each citation resolves to the revision used for the answer.

The tradeoff

Vector search adds ingestion and evaluation work. If the source collection is small and well structured, ordinary search with direct links may be a better first product. A retrieval benchmark should justify the extra system.