What did the 2020 paper actually build?

Lewis, Perez, Piktus, and colleagues introduced retrieval-augmented generation in Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. The parametric memory was a pre-trained BART sequence-to-sequence model.

The non-parametric memory was a dense index of a December 2018 Wikipedia dump, split into about 21 million 100-word chunks and searched with a Dense Passage Retriever.

They trained the query encoder and the generator together. The document index stayed fixed.

The point of the hybrid was that world knowledge could be inspected and replaced without retraining the whole generator.

On their tests, the models beat a BART baseline that had no retrieval, and they reported more specific and more factual generation on the tasks they measured. Those results belong to that setup.

They are not a score for a chatbot product.

How does RAG work?

The paper compares two ways of using the retrieved passages. RAG-Sequence uses the same passage for every token of the answer.

RAG-Token can use a different passage for each token, then combines those choices.

  1. Encode the question and search the index.
  2. Take the top passages.
  3. Generate an answer conditioned on the question and those passages.
  4. Read the passage against the sentence that cites it.

Where can a RAG answer go wrong?

A later product that searches your files and pastes chunks into a prompt is using the same idea, not the paper’s training recipe. The chain can still fail.

Our guide to AI hallucinations is the reason a fluent answer is not evidence, and how to check an AI citation is the routine for the passage itself.

Three places to check
StagePossible problemCheck
DocumentAn outdated release noteSource date and current version
PassageA missing regional exceptionThe surrounding context
AnswerA claim the citation does not supportThe sentence against the passage

When is fine-tuning the other option?

Retrieval changes what the model can see for one question. Fine-tuning changes the model.

The choice depends on whether the failure is missing information or unwanted behavior. The comparison is in RAG vs fine-tuning.

Common questions

What is agentic RAG?

It is a label for a system that decides what to retrieve, and sometimes whether to retrieve again, instead of running one fixed search. The 2020 paper used one retrieval step.

Extra searches do not remove the need to read the passages.

Does RAG eliminate incorrect answers?

No. The wrong document, a cut-off passage, or a sentence that does not follow the passage can each produce a wrong answer.

Does a citation prove a claim?

No. Check that the cited passage supports the claim as written.

Sources & further reading

  1. Lewis et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020) ↗
How this story was made

Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. Cover photography is illustrative and does not depict the specific announcement or product.

Our editorial standards
Back to all stories