Vector multi-tenancy fails at metadata isolation before retrieval relevance

Vector multi-tenancy fails at metadata isolation before retrieval relevance. A nearest-neighbour query can return semantically perfect passages and still be catastrophically wrong if one belongs to another tenant. Relevance metrics reward that result; the authorization model must forbid it. I therefore treat tenant identity as part of the index contract, not as an optional filter supplied by application code.

The common pool model stores many tenants in one collection and adds a tenant_id payload. It is operationally attractive: fewer collections, better resource utilization, simpler index management. Its failure mode is equally simple. One caller omits the filter, composes it with an unsafe OR, or trusts a tenant identifier supplied in the request, and the search space becomes everybody’s data. Embedding distance has no concept of ownership.

This is why vector search is an indexing decision with security consequences. Pre-filtering constrains the candidate set before approximate search; post-filtering retrieves broadly and removes disallowed matches afterward. Those approaches differ in recall and performance, but for tenancy they also differ in exposure. A system that searches unauthorized vectors and merely hides them later may still leak through scores, timing, logs, caches, or downstream rerankers.

Isolation models form a spectrum. Collection-per-tenant gives a clear silo and straightforward deletion, but large tenant counts create provisioning and index overhead. Namespaces and per-tenant shards split the middle: Pinecone namespaces and Weaviate tenant shards make tenant selection a structural operation rather than an arbitrary predicate. Payload partitioning offers the best density, but only if the retrieval service injects a mandatory tenant filter derived from trusted identity and callers cannot override it.

The word “metadata” understates what is carried there. Tenant, user, purpose, document ACL, residency, retention, consent, source identity, and embedding-model version can all determine whether a vector is eligible. Data rights belong in lineage, and retrieval is one place those rights become executable. If ingestion drops the metadata, no clever query planner can reconstruct authority later.

Federated agent systems make the boundary harder. Capability matching may need private set intersection; selective disclosure may reveal that an agent is qualified without revealing its full profile; a trusted mediator may plan which role can see which data. Those mechanisms exist because multi-agent communication edges combine useful context with privacy risk. A vector store that collapses all embeddings into one ungoverned similarity space defeats that architecture at the first lookup.

Testing must target the isolation mechanism, not just positive retrieval. I want adversarial cases that omit tenant context, substitute another tenant ID, combine filters, exercise caches, query during tenant deletion, and rerun after an embedding migration. The invariant is not “no foreign document appears in the final answer.” It is “no unauthorized document becomes a candidate, leaves evidence in a trace, influences a score, or survives beyond its retention boundary.”

There is one precise concession: collection-per-tenant can be the right default for a small number of high-value tenants with strict isolation requirements. Paying extra storage and operations is rational when the blast radius justifies a hard silo. The claim is not that pooled indexes are inherently unsafe; it is that their efficiency depends on mandatory, identity-derived metadata enforcement at every read and write path.

Agent tenancy starts with state isolation, and vector state is no exception. Choose the silo, namespace, shard, or pool deliberately. Then make tenant identity impossible to omit, propagate rights through ingestion and deletion, and measure isolation failures separately from recall. A retrieval system that finds the right passage for the wrong customer is not almost correct. It is a security incident with an excellent relevance score.