Routing tables are policy interfaces, not infrastructure configuration

I treat a routing table as a policy interface, not infrastructure configuration. The moment a route chooses one model, tool server, or agent over another, it allocates capability, cost, latency, and authority. Calling that choice plumbing hides the people affected by it and lowers the standard for changing it. A route is where an architectural preference becomes operational behavior.

The Listener–Target–Route model makes the separation useful. A listener defines how traffic enters: protocol, endpoint, hostname, and inbound authentication. A target defines where work can go: an LLM provider, MCP server, API, function, or another gateway. The route joins them through a forwarding rule. That join is not neutral. Its match conditions, target eligibility, ordering, and fallback chain decide which requests receive which service under which constraints.

I therefore want a route to expose policy in terms an operator can review. The contract should name the principal and tenant, request class, required capability, eligible targets, budget state, latency objective, trust rule, content checks, and failure behavior. Cost-aware routing may move routine work to a cheaper model as spend approaches a threshold. Capability-aware routing must exclude targets that cannot support vision or tool use. Latency-aware routing may shift traffic when recent response time deteriorates. These are competing priorities made explicit at one decision boundary.

Policy belongs at that boundary because AI gateways carry several kinds of traffic at once. LLM requests, MCP tool calls, agent-to-agent delegations, and ordinary APIs may all cross the same control point. A unified policy engine can attach rate limits, budget enforcement, prompt inspection, and redaction to routes instead of rebuilding them for every backend. One organization-wide token allowance only means something if consumption is aggregated across every provider the organization can reach.

This framing changes how I manage routing changes. A new target is an inventory change; making it eligible for regulated data is a policy change. A new route is not complete when its syntax validates. It needs representative decision fixtures, explicit precedence, a defined no-match outcome, fallback limits, and evidence that denied traffic stays denied. I want a semantic diff that says which principals or request classes gain and lose reachability, not merely which YAML lines moved.

The control-plane and data-plane split supports this lifecycle. The control plane can watch declarative configuration, validate and distribute a new decision graph, while a lightweight data plane routes, translates protocols, enforces policy, and emits traces on the request path. That separation makes rapid reconfiguration possible without turning every application release into a routing release. It also creates an obligation: every data-plane decision should identify the route version and policy version it actually applied.

Observability should reconstruct decisions rather than celebrate throughput. For each request, I want the matched route, rejected alternatives, selected target, budget and health inputs, fallback transitions, credential scope, policy result, and final outcome correlated by request or agent identity. RED metrics still matter, but aggregate rate, errors, and duration cannot explain why one tenant was downgraded, why a tool call crossed a trust boundary, or whether a fallback silently widened authority.

There is one precise concession: a static internal proxy with one listener, one target, no sensitive data, and no differential access can keep its route as ordinary configuration. In that bounded case, a policy interface would add ceremony without changing a decision. The boundary ends when target selection varies by identity, capability, budget, health, content, or trust, because the table then governs outcomes rather than merely wiring endpoints.

I would expose routing through versioned, testable policy-as-code and an operator surface that shows effective behavior before promotion. The table should answer who can reach what, why this target won, what happens when it fails, and which change produced the decision. Infrastructure still executes the route. Policy is what the route means. Treating those as separate concerns is how I keep a convenient forwarding rule from becoming an invisible allocation of cost, reliability, and authority.