Episodic memory retrieval needs SQL and semantics, not embeddings alone

Episodic memory retrieval needs SQL and semantics, not embeddings alone. An episode is not merely text that resembles the current prompt. It is an event involving a particular actor, task, time, tenant, outcome, and authority boundary. Those fields belong in a query plan before vector similarity gets a vote.

Consider a support agent asked whether this customer has complained about a delayed refund. Semantic search may find documents about refunds, delays, and frustration. The operational question is narrower: this customer, complaints rather than policy pages, within a relevant interval, perhaps excluding retracted or unauthorized records. SQL is good at those predicates because identity, recency, status, and scope are exact constraints, not shades of meaning.

Embeddings remain valuable after that reduction. Users rarely repeat the same wording, and episodes often encode meaning in narrative form. “The money still has not arrived” should match a delayed-refund concept even if the phrase never appears in a canonical taxonomy. Semantic retrieval can rank the eligible episodes by intent, theme, and contextual resemblance. Its job is to order plausible records, not decide which records the agent is allowed to see.

I model the retrieval path as stages. First bind tenant, subject, task, permissions, and temporal window. Then apply lifecycle state such as active, superseded, disputed, or deleted. Next run lexical and semantic candidate generation over the surviving set. Finally rerank with task-specific evidence and return provenance with each result. Memory reads are query plans because the order of these operations changes correctness as well as speed.

Time deserves special treatment. Episodic questions often mean “what happened most recently,” “what preceded this failure,” or “what was believed when the decision was made.” A vector score cannot reliably express interval containment, ordering, or valid-time versus transaction-time distinctions. Bi-temporal records let the system reconstruct both the world an event described and the history of what the agent knew.

The combined design also makes failures diagnosable. If the correct episode never entered the candidate set, inspect scope, filters, ingestion, and indexing. If it entered but ranked badly, inspect semantic representations and reranking. If an obsolete episode won, inspect lifecycle and temporal logic. Collapsing everything into one similarity score turns these distinct defects into “retrieval quality,” which is too vague to operate.

This architecture changes storage choices. The primary record needs typed columns and durable identifiers; text and embeddings are derived indexes over that record. A vector database can participate, but it should not become the source of truth for consent, deletion, tenant boundaries, or event order. Vector search is an indexing decision, not permission to discard the data model.

There is one precise concession: for a small, single-user notebook with no regulated data and no meaningful identity or temporal ambiguity, embedding-only retrieval may be the cheapest useful starting point. The operational cost of a richer schema can exceed the harm from occasional imprecision. The boundary moves once episodes drive external actions, cross users, or must survive audit and deletion requests.

I want episodic memory to answer two questions separately: which records are eligible, and which eligible records are useful now. SQL establishes eligibility; semantic ranking estimates usefulness. Keeping that division explicit produces retrieval that is more accurate, more governable, and far easier to debug than a nearest-neighbour query pretending to be memory. It also gives operators a concrete explanation for why an episode appeared.