Retrieval-Augmented Generation

What it is

Retrieval-augmented generation searches a document set for relevant passages and gives them to a language model as the basis for its answer. The term was introduced by Lewis and colleagues in 2020, though what that paper described is narrower than what now carries the name: a fine-tuning recipe in which the generator and the retriever are learned jointly against a dense vector index of Wikipedia. Systems built under the name today usually leave the language model itself untrained — they index a corpus, retrieve, and put the passages in the prompt at question time; where training does happen it is on the retrieval side, such as fine-tuning the embedding model on a domain corpus. What survived from the paper is the property it demonstrated: the index can be swapped to update what the system knows without retraining the model, which is why retrieval became the standard way to answer questions about a company's own documents.

Why it matters

RAG relocates the accuracy problem rather than solving it. The model is now only as reliable as the passages retrieved, and those are only as reliable as the corpus they came from — so an unreconciled document set produces an unreliable system no matter how good the retrieval or the model. Teams routinely tune chunking and rerankers for months while the underlying corpus still contains three different answers to the same question. Knowledge Company works on the corpus, because it sets the ceiling the other two cannot pass.