Memory writes are schema decisions; memory reads are query plans

Every agent memory system answers two questions: what gets stored when the agent learns something, and what comes back when the agent needs it. Teams obsess over the second and improvise the first. That's backwards. Retrieval quality is mostly decided at write time — a read can only be as good as the record it finds.

Writes are where the design lives

Compare the systems that actually work in production. Mem0 runs every conversation turn through an extract-then-update pipeline: distill candidate facts, then decide per fact whether to add, update, delete, or do nothing against the existing store. Zep's Graphiti builds a temporal knowledge graph — episodes, entities, edges — and invalidates edges when new information contradicts old. Letta (MemGPT) makes the agent itself manage a typed memory hierarchy. Three different architectures, one shared conviction: a memory is a structured record with a lifecycle, not a transcript chunk with an embedding.

The write-side decisions are schema decisions in the plain database sense: fact distillation versus verbatim append is a grain decision. Bi-temporal fields (when it happened vs. when we learned it) are a versioning decision. Entity merging and edge invalidation are integrity constraints — and note that the merge-versus-create call is itself governed by a tunable similarity threshold, which makes your dedup policy a numeric parameter someone must own, not an emergent behavior. Importance-weighted eviction is a retention policy. None of this is exotic — it's data modeling wearing an AI costume, again.

A schema comparison across these frameworks also exposes the division of labor that makes writes safe: the LLM decides content; deterministic code decides serialization. Graphiti won't let the model write arbitrary graph queries — extraction output is bound into predefined, parameterized Cypher. LangMem wraps whatever the model extracted in a UUID-keyed, append-only record envelope the model never touches. The model's creativity is confined to what a memory says; where and how it lands is code. Teams that let the model improvise the write path are trusting a stochastic process with referential integrity, and it shows up later as memories nothing can find.

Reads are query plans, not similarity searches

Once the store is typed, a memory read stops being "embed the query, take the top-k" and starts looking like what a query planner does:

1. Push predicates down first. Tenant, user, memory type, time window — metadata filtering before vector search, not after. Filtering a candidate set that similarity already chose is how cross-user leakage and stale-fact retrieval happen.
2. Choose the access path by corpus size. Below a few tens of thousands of records, brute-force scan beats an ANN index on both recall and operational simplicity. HNSW and IVF-PQ are optimizations you graduate into, not defaults. Most per-user agent memories never earn an index.
3. Rerank as a second stage. Embeddings recall candidates; a cross-encoder or the structure itself decides what deserves the context window. Graph stores make this concrete: Zep reranks by node distance from the conversation's center and by how often an episode mentions the entity — relevance signals that exist only because the write path built structure to compute them over. Trusting cosine similarity alone is trusting a lossy index to make a relevance judgment.
4. Plan for write-to-read lag. Extraction, entity resolution, and indexing take time, so a memory written this turn may not be reliably retrievable next turn. That's a consistency model, not a bug — decide explicitly whether the current session needs read-your-writes (keep fresh facts pinned in context) or can tolerate eventual visibility.

The numbers back the structure

The benchmark results are striking mostly for what they say about token economics. On LongMemEval, Zep's graph memory beat full-context GPT-4 by around eight points — while a structured store like Mem0 holds roughly 7k tokens per user against 26k for stuffing the full history into context. One nuance keeps the claim honest: the structured store can be far larger at rest than the transcript it came from — Zep's own case study measured a ~600k-token graph built from a 26k-token conversation, because structure multiplies representation. The economics work anyway, since what matters is tokens per read, not bytes at rest: the write pipeline pays a storage premium so every retrieval returns a small, dense, relevant slice. Structured memory isn't compression at rest — it's compression at read time, with judgment.

The uncomfortable conclusion for anyone shopping for a "memory layer": the vendor choice matters less than your record design. A well-typed memory schema on a plain table with brute-force search will outperform an untyped pile of embeddings on the fanciest vector database, every time.