Guide · 11 chapters · 1 live

Production RAGThe complete guide

How retrieval-augmented generation actually works — and how to make it reliable in production. Measured, not hand-waved.

Start with Chapter 1 →Get new chapters (RSS)
Before you start

How to read this

Who it's for

Backend and AI engineers building search or chat over their own data — or anyone who wants to know why their RAG answers are wrong.

How it's written

Real-life analogies first, then diagrams you can play with, then short code. Every claim that can be measured, is.

Two ways in

Read in order to build the full mental model — or jump to the stage of the pipeline that's breaking for you.

The map

The RAG pipeline

Every RAG system does two jobs: index your documents ahead of time, then answer each question by retrieving the right passages and handing them to an LLM. Each stage below links to the chapter that covers it.

Indexing · offline

  1. Documents
  2. Chunksplit docs into passagesCh 3
  3. Embedpassages → vectorsCh 1Ch 2
  4. Storevector index + metadataCh 5

Answering · per question

  1. Question
  2. Embed queryquestion → vectorCh 1Ch 2
  3. Retrievenearest k passagesCh 6Ch 7Ch 11
  4. Rerankre-order the best fewCh 8
  5. GenerateLLM answers with context

Across every stage: evaluate (Ch 4) and operate — debugging, cost (Ch 9–10).

Contents

Chapters

  1. 01How vector search works: embeddings, similarity, and vector databases explainedWhy keyword search fails, what an embedding is, how ANN indexes find neighbours fast, and when you actually need a vector database.11 min
  2. 02Choosing an embedding model: OpenAI vs open-source, benchmarkedDimensions, cost, latency, and retrieval quality — and why switching models later is a migration.Soon
  3. 03Chunking strategies for RAG: fixed-size vs recursive vs semantic (benchmarked)The least glamorous decision that quietly decides whether retrieval returns the right thing.Soon
  4. 04How to evaluate RAG retrieval quality (the metrics that matter)recall@k, MRR, and faithfulness — so every later change is measured, not guessed.Soon
  5. 05pgvector vs a dedicated vector database: when do you actually need one?The storage decision teams get wrong, by data size, filtering needs, and operations.Soon
  6. 06Metadata filtering in RAG: the cheap accuracy win most people skipPre-filter, post-filter, and filtered search — and the trap in between.Soon
  7. 07Hybrid search in production: combining BM25 and vector searchWhy pure vector search misses exact matches like product codes, and how to fuse scores.Soon
  8. 08Does reranking actually improve RAG answers? (I measured it)Before-and-after numbers for adding a reranker to the pipeline.Soon
  9. 09Why your RAG returns irrelevant chunks (and how to debug it)A field guide to retrieval failure modes and how to find which one you have.Soon
  10. 10Cutting RAG costs: caching embeddings and LLM callsPractical cost engineering for embedding and generation.Soon
  11. 11Late-interaction retrieval (ColBERT): is it worth the complexity?An honest verdict on a powerful but heavier retrieval approach.Soon
Pick your path

By use case

Reference

Glossary

Embedding
A list of numbers representing the meaning of a piece of text. Similar meaning → nearby numbers.
Cosine similarity
How closely two embeddings point in the same direction. 1 = same meaning, ~0 = unrelated.
ANN
Approximate nearest neighbour search: finds probably-the-closest vectors without checking every one.
HNSW
A popular ANN index: a layered graph you search from coarse to fine, like motorways to streets.
IVF
An ANN index that groups vectors into clusters and only searches the nearest few.
Chunk
A passage cut from a document; the unit you embed, store, and retrieve.
recall@k
Of the passages that should have been retrieved, what share made it into the top k.
Reranker
A slower, more accurate model that re-orders the top candidates after retrieval.
Hybrid search
Combining keyword (BM25) and vector search so exact terms and meaning both count.