Cache keys are behavior contracts, not performance hints

A cache key in an agent system is a behavior contract, not a performance hint. It declares which requests are equivalent enough to share a prior computation. If that declaration omits a variable that changes the answer, policy, tool surface, or permitted side effect, the cache does not merely become inaccurate — it causes the system to execute under the wrong conditions.

Prompt caching makes the point visible. Static system instructions and tool definitions can form a reusable prefix, delivering substantial cost and latency reductions when the bytes remain stable. A single changed tool description or reordered instruction can invalidate that prefix. That sounds like an efficiency concern, but the inverse is more important: if an application-level cache continues to hit while those behavioral inputs changed, it can return an answer produced under an obsolete contract.

I therefore design cache identity from the decision boundary outward. Model and version belong in the key. So do the normalized system prompt, tool-manifest digest, policy version, tenant, data snapshot or freshness epoch, retrieval configuration, locale, and any user attribute that changes authorization or semantics. The exact set varies, but the test does not: could changing this value legitimately change what the system is allowed to conclude or do? If yes, it belongs in identity or must force invalidation.

This is why semantic caching can lie. Similar wording is not equivalent intent, and equivalent intent is not equivalent authority. “Show my invoices” and “show Acme’s invoices” may sit close in embedding space while crossing a tenant boundary. A similarity threshold cannot carry the security meaning that an explicit tenant and policy dimension carry. Semantic lookup may propose candidates; deterministic constraints must decide eligibility.

The contract also needs an observable version. A hit record should state the complete key components, the artifact that produced the cached value, its age, and the reason it remained eligible. That makes cache behavior replayable rather than magical. When an agent release versions model, policy, tools, and state together, the cache namespace should move with that release unless compatibility has been demonstrated.

Good key design exposes operational trade-offs instead of hiding them. Fine-grained keys lower the hit rate but preserve distinctions. Coarse keys improve reuse but merge contexts. Cost is an architectural property, yet a cheaper wrong answer is not optimization. I would rather measure why a key fragments than increase reuse by deleting dimensions whose behavioral significance has not been tested.

Invalidation deserves the same rigor as construction. Policy revocation, source correction, model retirement, schema migration, and changed data entitlements should emit explicit invalidation events. Time-to-live is only a freshness ceiling; it cannot express why an artifact became unsafe before its clock expired. A cache that knows when to stop answering is part of the control plane.

There is one precise concession: a byte-identical provider-managed prefix cache for immutable public instructions can safely use a narrow identity because it reuses computation, not a completed answer or authorized action. Even there, the boundary must include every prefix byte and tool definition the provider treats as cached. The concession does not extend to semantic response caches, retrieval results, or effectful tool outcomes.

The practical review is simple. Enumerate every input that can change meaning, evidence, authority, or effect; map each to the key, an invalidation event, or an explicit proof of irrelevance. Then test cross-tenant, cross-policy, stale-data, and tool-version cases as correctness failures. Pinned tool schemas make one of those dimensions stable, but stability is never implied. A cache key is the executable claim that two situations are interchangeable. Production systems should make that claim deliberately.