Embedding migrations need dual-read cutovers, not one-shot reindexing

Embedding migrations need dual-read cutovers, not one-shot reindexing. A new embedding model does not merely produce better numbers in the same index; it creates a different retrieval system with a different vector space, ranking behaviour, storage footprint, latency profile, and failure distribution. Rebuilding the corpus and switching an alias at midnight treats that behavioural change like a file copy. I treat it like a production data migration.

The incompatibility is fundamental. Vectors from different models cannot be compared meaningfully just because they share a distance metric. Even models with the same dimension encode neighbourhoods differently, while a dimension change also alters storage, memory pressure, and index parameters. Query embeddings and document embeddings must therefore move as a versioned pair. The model identifier, preprocessing rules, chunking version, distance function, and index build parameters belong in the index contract—not in a deployment note someone hopes remains accurate.

I start by building a shadow index from a stable source snapshot while the old index continues serving. New or changed records are then written to both versions, with an ordered change log closing the gap between the snapshot and live state. Each write carries source identity and embedding version so reconciliation can prove coverage rather than infer it from equal row counts. This is the same reason RAG must be operated as a data pipeline: derived vectors are reproducible state, and reproducibility depends on source lineage.

Dual-read is the verification phase. Production queries go to the established path and, in shadow, to the candidate path. I preserve both ranked lists, metadata filters, scores, latency, and downstream answer evidence. Offline recall tests catch known cases; shadow traffic exposes actual vocabulary, tenant filters, rare identifiers, and long-tail query shapes. A migration earns promotion when slices improve or remain inside explicit tolerances—not when an average benchmark rises.

The comparison cannot rely on raw similarity scores because score distributions change across models. I compare outcomes: whether judged-relevant items appear, rank movement for protected query sets, filter correctness, empty-result rate, tail latency, and the answer failures attributable to retrieval. Retrieval and generation need separate diagnostics, otherwise a stronger generator can hide a weaker index during the cutover.

Rollback also needs a write path. If the new index fails after promotion, the old one must still contain every source change made during the trial. That means dual-write duration extends through a defined observation window, and retirement waits until replay, export, and repair have been tested. The alias switch is the least interesting part; synchronization, comparison, and reversibility are the migration.

Operational ownership should be explicit as well. The team changing the model owns the backfill manifest, mismatch alarms, comparison report, and rollback trigger. The application team owns the protected query set and acceptable behavioural deltas. Without that split, platform engineers can prove vector coverage while product failures remain invisible.

There is one precise concession: for a small, immutable corpus with no live writes and a low-consequence search surface, an offline rebuild followed by a measured acceptance test can be enough. Dual-read machinery has a cost. But once retrieval influences customer answers, agent memory, regulated evidence, or continuously changing data, the cost of an invisible regression is higher than the temporary duplicate compute.

Vector search is an indexing decision, and indexes have lifecycle obligations. I want embedding releases to declare compatibility, shadow against real traffic, preserve a reversible source-to-vector lineage, and cut over only through evidence. One-shot reindexing proves that the job finished. Dual-read cutover proves that the system still works.