Loading
Retrieval-Augmented Generation (RAG) is an architectural pattern that pairs a generative AI model with a retrieval system, so the model's answers are grounded in passages pulled from an external knowledge source at query time rather than relying only on what was encoded into its training data. A retriever converts the user's query into an embedding, searches an index of documents (commonly a vector database) for the most relevant passages, and feeds those passages to the generator as additional context before it produces a response, typically with citations back to the source material. RAG has become the dominant pattern for deploying large language models against proprietary, confidential, or frequently changing enterprise knowledge, because it lets an organization update what a model effectively "knows" by updating a document store rather than retraining the model. For AI governance teams, RAG matters because it materially reduces hallucination and supports source citation and auditability, while introducing its own risk surface, including indirect prompt injection via retrieved content, document-level access control, and retrieval-corpus poisoning, that frameworks like NIST AI 600-1 and the OWASP Top 10 for LLM Applications now address directly.
Run the free AI Health CheckRetrieval-Augmented Generation (RAG), an architectural pattern that combines a generative AI model with a retrieval system, so the model generates responses grounded in retrieved documents rather than only its parametric training data.
RAG is the most common pattern for deploying LLMs against enterprise knowledge, internal policies, technical documentation, customer history, regulatory text. It substantially reduces hallucination by grounding outputs in retrievable source documents, supports citations, and allows knowledge updates without retraining. RAG does not eliminate hallucination, and creates its own governance considerations around document access control, source quality, and prompt injection via retrieved content.
Source: NIST AI 600-1; OWASP LLM Top 10 (LLM01 Prompt Injection)
A RAG system is built in two stages. First, in an offline ingestion stage, source documents are split into chunks, each chunk is converted into a numerical vector (an embedding) by an embedding model, and the vectors are stored in a searchable index, usually a purpose-built vector database. Second, at query time, the user's question is embedded using the same model, the index is searched for the chunks whose vectors are most semantically similar to the query (nearest-neighbor search), and the top-ranked chunks are inserted into the prompt alongside the original question. The generative model then produces its answer conditioned on both the query and the retrieved passages, which is what allows it to cite or quote specific source material rather than reconstructing an answer purely from patterns memorized during training.
This design directly reflects the original framing from Lewis et al.'s 2020 paper, which described RAG models as combining a pre-trained model's "parametric" memory (knowledge encoded in its weights) with a "non-parametric" memory (an external, updatable index of text that can be swapped or refreshed without touching the model). That is the practical reason RAG is attractive for governance: swapping, correcting, or deleting a document in the index changes what the system can say next, in a way that is visible, reversible, and auditable, unlike a change to model weights.
Reduced (not eliminated) hallucination
Grounding responses in retrieved text lowers the rate of fabricated facts compared to an ungrounded model, because the generator has real source material to draw from, but it can still misinterpret, blend, or contradict that material.
Citability and provenance
Because each answer can be traced to specific retrieved chunks, RAG systems can show users which document supported a given statement, which supports explainability and audit requirements that purely parametric models cannot easily meet.
Knowledge updates without retraining
Adding, correcting, or removing a document in the retrieval index changes model output immediately and reversibly, avoiding the cost, delay, and opacity of retraining or fine-tuning a model to reflect new facts.
A new governance boundary, not a solved problem
RAG shifts risk rather than removing it: instead of only governing the model, organizations must now govern the retrieval corpus, the access controls on it, and the pipeline that connects retrieved content back into the model's context window.
Indirect prompt injection (OWASP LLM01: Prompt Injection)
Because LLMs process instructions and retrieved data through the same channel, an attacker can hide malicious instructions inside a document, webpage, or file that later gets retrieved and fed to the model, which may follow those hidden instructions as if they came from the user. OWASP's Top 10 for LLM Applications (2025) ranks Prompt Injection as its top risk for the second consecutive edition, and explicitly distinguishes this indirect, retrieval-borne form from direct user-typed injection.
Vector and embedding weaknesses (OWASP LLM08)
The 2025 OWASP Top 10 for LLM Applications added a category dedicated to weaknesses in how vectors and embeddings are generated, stored, and retrieved, covering risks such as embedding inversion, cross-tenant data leakage in shared vector stores, and manipulation of the retrieval index itself.
Retrieval-corpus and knowledge-base poisoning (NIST AI 600-1)
NIST's Generative AI Profile discusses data and retrieval poisoning as a risk requiring attention: if the document corpus or vector store behind a RAG system is compromised, the model's grounded answers become grounded in bad information, so NIST recommends the ability to revert to a known-good baseline and to treat the corpus itself as part of the security perimeter.
Document-level access control and oversharing
A RAG system that retrieves across a shared corpus can surface a document, or a snippet of one, to a user who would not otherwise have permission to see it, unless retrieval is filtered by the same access controls that govern the underlying documents, a common real-world gap in early enterprise RAG deployments.
Teams grounding an LLM in domain-specific knowledge generally choose among three approaches. Fine-tuning retrains model weights on a labeled or curated dataset, which can teach a model a style, format, or narrow skill well but is comparatively costly, slow to update, and hard to audit for exactly why the model produced a given output. Long-context prompting simply pastes large amounts of source text directly into the prompt, which is simple but expensive at scale and still requires some way to select which text is relevant. RAG sits between these: it keeps the model's weights frozen and its cost bounded by retrieving only the most relevant chunks for each query, at the expense of added system complexity, an ingestion pipeline, an index, and a retrieval step that all need to be built, secured, and maintained. In practice, many production systems combine RAG with light fine-tuning of the retriever or embedding model, rather than treating the approaches as mutually exclusive.
Does RAG eliminate hallucination?
No. RAG substantially reduces hallucination by grounding responses in retrieved text, but it does not eliminate it, a model can still misread, over-generalize from, ignore, or blend retrieved passages incorrectly. NIST AI 600-1 addresses this residual risk under its "confabulation" and "information integrity" risk categories and recommends verification, citation, and human review rather than treating retrieval as a guarantee of accuracy.
What is the difference between RAG and fine-tuning?
Fine-tuning updates a model's internal weights using a training dataset, which is slower, costlier, and harder to audit or reverse. RAG leaves the model's weights untouched and instead supplies relevant external context at the moment of the query, so organizations can update, correct, or remove source documents, and immediately change model behavior, without retraining anything.
Do you need a vector database to build RAG?
Most production RAG systems use a vector database or vector index for approximate nearest-neighbor search over document embeddings, since it scales well to large, semantically diverse corpora. However, retrieval can also be done with keyword search (e.g., BM25), hybrid keyword-plus-vector search, or structured lookups against databases and APIs, the defining feature of RAG is that retrieval happens, not the specific retrieval technology.
What are the main security risks specific to RAG systems?
The OWASP Top 10 for LLM Applications (2025) identifies indirect prompt injection (LLM01: Prompt Injection), malicious instructions hidden inside retrieved documents, web pages, or files that the model treats as trusted input, as a risk especially relevant to RAG. It also lists Vector and Embedding Weaknesses (LLM08) as a distinct category covering attacks on the embedding and vector-store layer that RAG architectures depend on, such as embedding inversion or corpus poisoning.
Who invented RAG and where does the term come from?
The term and architecture were introduced by Patrick Lewis and co-authors (then at Facebook AI Research, now Meta AI) in the 2020 paper "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks," presented at NeurIPS 2020. The paper framed RAG as combining a pre-trained model's "parametric" memory with an external, updatable "non-parametric" memory accessed via a neural retriever.
Does RAG require retraining the underlying language model?
No, that is RAG's core advantage. The generative model's weights are not changed; retrieved context is injected into the prompt at inference time. Some teams separately fine-tune the smaller retriever or embedding model to improve retrieval quality, but this is optional and distinct from retraining the generator itself.
Last reviewed July 2026