RAG is a data pipeline with a retrieval step
Every RAG system is two pipelines: an indexing pipeline that turns documents into retrievable chunks, and a query pipeline that finds them. All the glamour lives in the second — CRAG, Self-RAG, GraphRAG, HyDE, rerankers, fusion strategies. Most of the failures live in the first. When a RAG system gives a wrong answer in production, my default diagnosis is now ingestion, not retrieval: what got indexed, how it was cut, what metadata it carried, and when it was last refreshed.
The reasoning is mechanical. The retriever can only rank what the index contains. If a policy document was split mid-table by fixed-size chunking, no reranker reassembles it. If the chunk lacks the metadata that says which product version it describes, no query rewrite disambiguates it. If the source changed last Tuesday and the index refresh is monthly, the most sophisticated agentic retrieval loop confidently returns last month's truth. These are not retrieval defects. They are data-quality defects, and they demand the unglamorous machinery data engineers have run for decades: transformation logic, freshness SLAs, metadata contracts, lifecycle management — the same discipline that makes data AI-ready in the first place.
Where the actual engineering is
Look at what the ingestion side genuinely involves. Chunking alone is a design space — fixed-size, recursive-character, semantic, sentence-window, parent-document hierarchies, late chunking that embeds the whole document before pooling passages — and the right choice depends on document structure, not fashion. Chunk metadata enrichment determines whether you can filter before you rank. Index freshness is a real strategy decision: full re-index versus incremental, and the shadow-index pattern for upgrading embedding models with zero downtime — because swapping embedding models mid-flight otherwise means a window where queries and documents live in different vector spaces. That failure even has a name: embedding drift, and its signature is exactly why it's dangerous — retrieval quality degrades silently, with no error, no exception, just gradually worse answers.
The evaluation literature quietly agrees. RAGAS separates context recall and context precision from answer quality for a reason: when context recall is bad, the generator never had a chance, and context recall is largely a function of what the indexing pipeline produced. Teams that only measure end-to-end answer quality end up tuning the reranker to compensate for a corrupted index — polishing the last mile of a road that's washed out upstream.
The concession: sometimes it really is the retriever. Pure dense retrieval reliably fumbles exact identifiers — part numbers, error codes, function names — and the fix genuinely is hybrid search, BM25 alongside embeddings with reciprocal rank fusion. Cross-encoder reranking buys real precision. Over-retrieval causing context rot is a query-side failure with a query-side fix. I'm not arguing the retrieval layer is trivial; I'm arguing about base rates and debugging order. And notice that even the hybrid-search fix is half an ingestion decision — an inverted index is something you build and maintain, not something you toggle.
So the operational advice is a debugging order. When answers are wrong, first pull the retrieved chunks and read them like a data engineer doing a quality audit: Is the fact in the index at all? Is it split across chunk boundaries? Is the metadata right? Is it stale? Only when the right chunks exist and don't come back should you touch the retriever. Teams that treat RAG as a model problem hire for prompts and rerankers. Teams that treat it as a data pipeline with a retrieval step hire for pipelines — and their systems keep working in month six, when the corpus has changed underneath everyone.