The operating system is the right mental model for agent runtimes

Of all the analogies the agent era has produced, one is load-bearing: the agent runtime is an operating system, with the model as its CPU. I mean this as an engineering guide, not a slogan — because the analogy doesn't just describe what we've built, it predicts what we'll need to build next, and so far its predictions keep landing.

Map the components and the correspondence is uncomfortably exact. The context window is RAM: fast, expensive, fixed-size working memory. Long-term stores — files, databases, vector indexes — are disk. The moment you accept that, compaction stops being a hack and becomes what it always was: paging, with a page-replacement policy you'd better choose deliberately. Tools are device drivers behind a syscall boundary — the model requests an action in a standard format, the runtime validates, dispatches, and returns a result, and the model no more executes the tool than a process executes the disk controller. Permission modes are user space versus kernel space: some operations execute freely, others trap into a supervisor — a human, a policy engine — before proceeding. The agent loop itself is a scheduler: it decides what runs next, enforces budgets on tokens and iterations, and preempts work that exceeds them. Sub-agents are processes with isolated address spaces. Even speculative execution has its analog in running parallel tool calls you might discard.

The payoff isn't the elegance; it's the imported discipline. Operating systems spent fifty years learning how to manage scarce, shared, failure-prone resources, and every lesson transfers as a requirements list. Processes need isolation — so sub-agents shouldn't share mutable context. Resources need quotas — so token budgets and loop-iteration caps are scheduler features, not afterthoughts. Syscalls need an audit trail — so tool dispatch is where logging and permissioning live. Memory needs a hierarchy with explicit movement policies — so "what stays in context" deserves the same rigor as a cache-eviction design. When I review an agent architecture, I now ask OS questions: where's your syscall boundary, what's your paging policy, who can preempt a runaway process? Teams that can't answer usually have an incident coming that the analogy would have predicted.

The analogy also explains the industry's shape. The runtime layer — not the model — is where durable engineering value accumulates, exactly as the OS mattered more than the instruction set architecture. Models are swappable CPUs; the scheduler, memory manager, and driver ecosystem are where switching costs and reliability actually live.

The concession is the place the analogy breaks, and it breaks in one precise spot: the kernel is stochastic. A real OS never misreads its own page table; an LLM misreads its context routinely, confabulates syscalls that don't exist, and returns different outputs for identical inputs. No operating system ever needed an eval suite, a verifier model, or a hallucination taxonomy, because determinism was the one property the whole edifice assumed. So the analogy is a design guide for the resource-management half of the problem and silent on the verification half — push it into "the kernel is trustworthy" territory and it will hurt you. The machinery that fills that gap — evals, verifiers, structured-output validation — has no OS ancestor. It's genuinely new.

Used within its limits, though, this is the rare analogy that generates architecture instead of decorating it. When someone asks me why an agent runtime needs budgets, isolation, audit, and paging policy, the answer fits in a sentence: because it's an operating system, and we already know what happens to operating systems that skip those.