Retrieval-Augmented Generation (RAG) Basics

Retrieval-augmented generation (RAG) helps an AI application answer questions using a selected collection of documents. The application finds relevant passages, places them with the question in the model's input, and asks for an answer based on that evidence. This does not change the model's training.

RAG is useful for product manuals, policies, and other information that may be private or change over time. It can reduce unsupported answers, but it cannot make a missing or outdated document correct.

How RAG works

1. Prepare: Split long documents into searchable passages and keep source details such as title and update date.
2. Retrieve: Search for passages related to the user's question.
3. Augment: Give the model the question and selected passages, with instructions to use them as evidence.
4. Generate: Produce an answer and identify which passage supports it when the application can do so.

Search can use keywords, vector similarity, or a combination. An embedding may find related wording, but a high similarity score does not prove a passage is true, current, or applicable to the question.

Follow a policy question

Suppose a shop has two policy passages. One says standard items can be returned within 30 days. The other says damaged items should be reported within 7 days. A customer asks about returning a standard item. The first passage applies; the second covers a different case.

Example:

Question: How long can I return a standard item?
Retrieved passage: Standard items can be returned within 30 days.
Answer: You can return a standard item within 30 days.
Source: Standard returns policy

The answer should not combine the 30-day and 7-day rules. If the application finds no applicable passage, it should say the available documents do not answer the question. If two current passages disagree, show the conflict for review.

Try a small retrieval example

The Python program below ranks two local passages by shared words. It demonstrates retrieval only: it does not call an AI model, use embeddings, or generate an answer. Run it to see which passage would be supplied as context.

Example:

import re

pages = {
    "Returns": "The return window is 30 days for standard items.",
    "Shipping": "Standard shipping takes 3 to 5 days.",
}
question = "What is the return window?"
words = lambda text: set(re.findall(r"[a-z]+", text.lower()))
query = words(question) - {"what", "is", "the"}

ranked = sorted(
    pages.items(),
    key=lambda page: len(query & words(page[1])),
    reverse=True,
)
print(ranked[0][0])
print(ranked[0][1])

Output:

Returns
The return window is 30 days for standard items.

The result is relevant to this exact wording. A question about the “refund period” might not match “return window.” That is a limitation of this simple word-overlap method, not a failure of all RAG systems.

Check the evidence before answering

Test answerable, unanswerable, and conflicting questions. Check retrieval and the final answer separately. Apply document permissions before retrieval so a user cannot receive content they are not allowed to see.

Note: RAG supplies information at request time. Fine-tuning changes a model's learned behavior through training. The two methods solve different problems and can be used separately or together.

Conclusion

RAG connects a question to relevant source material before generating an answer. Good results require useful search, current documents, correct access controls, and honest handling of missing or conflicting evidence.