Retrieval-augmented generation, or RAG, searches an approved knowledge collection and gives the relevant passages to a language model before it answers. It does not retrain the model or guarantee truth. It creates a controlled path from a user question to source evidence, which can improve freshness, traceability and domain relevance.
How a RAG system works
An ingestion process reads documents, preserves metadata and permissions, splits text into useful units and creates searchable representations. At query time, retrieval finds candidates, optional reranking selects the strongest evidence and the model answers under explicit instructions. The response should cite its sources and abstain when evidence is insufficient.
When RAG creates business value
Good cases include internal policy assistants, technical support, product documentation, sales enablement and contract or procedure discovery. RAG fits when knowledge changes often, lives across many documents and needs source-level access controls. Start with one expensive search-heavy workflow and a measurable target such as resolution time or answer accuracy.
When not to use it
Do not add RAG to deterministic calculations, structured database reports or processes where a normal search interface works better. It is also a poor starting point when documents are obsolete, ownership is unclear or permissions cannot be represented. For high-impact decisions, use human review and authoritative systems rather than treating generated text as approval.
Security, compliance and cost
Enforce authorization before retrieval, not only in the interface. Log source IDs and versions without unnecessarily storing sensitive prompts. Define retention, deletion and vendor boundaries; defend ingestion and retrieved content against prompt injection. Cost includes document processing, search, model calls, evaluation and ongoing content operations—not just tokens.
Common mistakes
- indexing every file without an owner or quality filter
- ignoring document permissions and tenant boundaries
- splitting content with one arbitrary chunk size
- testing only polished demo questions
- accepting fluent answers without source support
- using a larger model to hide poor retrieval
- never refreshing, deleting or versioning content
- launching without feedback, evaluation and fallback paths
Practical RAG checklist
- choose one workflow and baseline its current cost
- inventory sources, owners, sensitivity and update frequency
- preserve metadata, permissions and document versions
- build representative questions with expected evidence
- measure retrieval separately from answer quality
- require citations and an insufficient-evidence response
- test leakage, prompt injection and cross-tenant access
- monitor latency, cost, failed queries and user feedback
- define refresh and deletion service levels
- keep human review for consequential decisions
When hiring a technical person makes sense
Hire an AI engineer or technical lead when data crosses multiple systems, access rules matter, evaluation is unclear or the prototype must become a supported product. The work is mostly integration, retrieval quality, security and operations—not merely selecting a model. See how to build an assistant with internal documents and when agents are appropriate.
Final takeaway
RAG succeeds when it returns the right permitted evidence, explains its sources and fails safely. Explore my applied AI services or contact me to scope a measurable pilot.