85 agentic tools audited for security incidents · Talk to the lab →
MotionticBook a call
Glossary

What is retrieval-augmented generation (RAG)?

Retrieval-augmented generation (RAG) is a method in which a system fetches relevant passages from a company's own documents or data at query time and places them in the model's prompt, so answers draw on current, citable sources.

Checked

A language model only knows what was in its training data and what is in its prompt. RAG fills the prompt with the passages that matter for each request, pulled from the organisation's own documents, tickets or records, so the model answers from current facts and can point to where they came from.

The term comes from a 2020 paper by Patrick Lewis and co-authors, which combined a pre-trained generator, the parametric memory, with a dense vector index queried by a neural retriever, the non-parametric memory.

The pipeline

  • Ingest and chunk

    Documents are parsed, cleaned and split into passages, with titles, dates and permissions kept as metadata.

  • Embed and index

    Passages are embedded into a vector index and usually indexed for keyword search too.

  • Retrieve

    Each query runs hybrid search, filtered by what the user is allowed to see.

  • Rerank

    A reranker reorders the shortlist so the most relevant passages reach the model.

  • Generate with citations

    The model answers from the passages and cites them. Anthropic's search_result content blocks let Claude cite retrieved chunks the way it cites web results.

Build or use a managed service

The labs now sell managed retrieval. OpenAI offers a file_search tool in the Responses API over managed vector stores, Google launched File Search in the Gemini API on 2025-11-06 with chunking, embeddings and citations handled, and Vertex AI RAG Engine offers a managed store or a bring-your-own one. These cut setup time for small and medium document sets, at the cost of control over chunking and ranking.

A custom pipeline earns its place when permissions are complex, documents need special parsing, or evals show the managed defaults miss too often.

RAG inside agents

In an agent, retrieval becomes a tool the agent can call several times rather than a single step before the answer. The RAG vs agentic search page covers when an index helps and when file tools are enough, and the RAG vs fine-tuning page covers when to retrain instead.

How Motiontic's engineers apply it

Motiontic's engineers build the eval set before the pipeline: real questions, the passage that answers each one, and the answer expected. Retrieval and generation are scored separately, so a wrong answer can be traced to a missed passage, a bad ranking or a model that ignored good context.

Where RAG pipelines fail, and the usual fix

Where RAG pipelines fail, and the usual fix
StageCommon failureUsual fix
ChunkingPassages lose the names and dates that made them findableContextual retrieval, better chunk boundaries
RetrievalExact codes or paraphrased questions missedHybrid search
RankingThe right passage is retrieved but ranked too lowA reranker
PermissionsUsers see passages they should notFilters applied at retrieval time
GenerationAnswer ignores or contradicts the passagesCitation requirements and graded evals

Common questions

What does RAG stand for?

Retrieval-augmented generation. The system retrieves relevant passages first, then the model generates an answer using them.

When is RAG not needed?

When the knowledge fits in the prompt. Anthropic's Contextual Retrieval post notes that a knowledge base under about 200,000 tokens can go straight into the prompt, with prompt caching to cut cost.

Is RAG the same as fine-tuning?

No. RAG changes what the model sees for each request. Fine-tuning changes the model's weights. RAG suits knowledge that changes; fine-tuning suits behaviour that should stay fixed.

Is RAG still relevant for AI agents?

Yes, as a tool. Agents call retrieval when they need it, sometimes several times per task, and indexed hybrid search remains the stronger option for large or conceptual document sets.

Running agents that have to hold up?

Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.