Question 1

What is RAG and how do you build it in n8n?

Accepted Answer

Definition plus the two-workflow structure — the second half is what sets the answer apart. "RAG grounds a model's answers in your own documents. A model cannot use knowledge absent from its training data — company policies, product docs — so RAG retrieves the relevant passages and hands them over as context. In n8n that means two separate workflows: Ingestion: I fetch the documents, split them with the Default Data Loader, turn them into vectors with an embedding model and write them via the Vector Store node's Insert Documents operation. I add metadata to each chunk — source, date — so I can filter later. Query: I attach the vector store to an agent as a tool and write its description. The agent decides which questions call for it; alternatively you use the node directly in the flow, and then it always searches. The most important technical detail: the same embedding model must be used on both sides. Otherwise the vectors do not line up and the results are nonsense — without throwing an error."

Question 2

A RAG system is answering incorrectly. How do you diagnose it?

Accepted Answer

Split the problem in two — that shows systematic thinking. "First I localise it: is the retrieval bad, or the answer? I look at the chunks the vector store returns. If the right passages come back, the problem is the prompt; if not, the problem is retrieval. Retrieval problems: 1. A different embedding model — if ingestion and query used different models the vectors do not line up. It is the most common cause, and changing the model means reindexing everything. 2. Chunk size — too large and the relevant sentence is buried in noise; too small and context is lost. Recursive splitting is the recommended strategy for most cases. 3. The limit — if too few chunks are returned, an answer spread across several documents comes back incomplete. 4. The document was never indexed — checking this feels banal, but it is often the actual cause. Answer problems: the right chunks arrive but the model uses them badly. Here I work on the prompt: an explicit "answer only from the provided context; if it is not there, say you do not know" prevents invented answers. In an agent setup there is one more cause: a vague tool description means the agent never consults the vector store and answers from the model's general knowledge instead."

Question 3

You are asked to build a Q&A system over internal company documents. Where do you start?

Accepted Answer

Start with the questions that come before the technical build — that is the experienced answer. "Before writing anything I ask four questions. 1. Which documents, and who keeps them current? A stale document means a wrong answer. If there is no update process, build that first — otherwise nobody trusts the system after three months. 2. Who is allowed to see what? HR documents are not open to everyone. You mark the source with metadata in the vector store and filter at query time. Adding that later is hard. 3. What does a wrong answer cost? A holiday policy and a legal document carry different risks. Where the risk is high, citing the source and being able to say "I don't know" are mandatory. 4. How many documents are there? A hundred pages and a hundred thousand call for different architectures. Then the technical part: I start small. Rather than indexing the whole corpus I take the single area that gets asked about most, build RAG over it and test with real questions. I tune the chunking strategy and the limit against those real questions. The most important prompt rule: if the answer is not in the context, the model must say so. A fabricated but convincing answer is the biggest risk in this kind of system — the user does not check it."

Component	Its role
Vector Store node	Stores and searches the vectors. Data is loaded with the Insert Documents operation.
Embedding model	Turns text into vectors. The same model must be used when inserting and when querying.
Default Data Loader	Splits the document into chunks
Text splitter	The chunking strategy: by character length, recursive (recommended), or by token count
Metadata	Extra information per chunk: source, date, category — for filtering later
Vector Store (as a tool)	Attaches to an agent; you set its description and result limit

RAG: AI over your own documents

📚 Sources and documentation