The operating system is the right mental model for agent runtimes

Of all the analogies the agent era has produced, one is load-bearing: the agent runtime is an operating system, with the model as its CPU. I mean this as an engineering guide, not a slogan — because the analogy doesn't just describe what we've built, it predicts what we'll need to build next, and so far its predictions keep landing.

Map the components and the correspondence is uncomfortably exact. The context window is scarce working memory: fast, expensive, and fixed-size. Files, databases, and vector indexes are durable backing stores, but copying their contents into the prompt is only one access path. Compaction is a lossy paging policy; a runtime can also keep a corpus outside model history and expose symbolic access to selected slices. One retained recursive-model design does this with metadata, a handle, bounded code, and persistent runtime state. I take it as a counterexample to prompt-only memory management, not as a prescription for a REPL in every agent. The reusable requirement is governed access to external state: the runtime decides which representation and operation may cross into model working memory. Tools remain device drivers behind a syscall boundary, permissions separate ordinary execution from supervised operations, the loop schedules bounded work, and sub-agents keep isolated address spaces.

The payoff isn't the elegance; it's the imported discipline. Operating systems spent fifty years learning how to manage scarce, shared, failure-prone resources, and every lesson transfers as a requirements list. Processes need isolation — so sub-agents shouldn't share mutable context. Resources need quotas — so token budgets and loop-iteration caps are scheduler features, not afterthoughts. Syscalls need an audit trail — so tool dispatch is where logging and permissioning live. Memory needs a hierarchy with explicit movement policies — so "what stays in context" deserves the same rigor as a cache-eviction design. When I review an agent architecture, I now ask OS questions: where's your syscall boundary, what's your paging policy, who can preempt a runaway process? Teams that can't answer usually have an incident coming that the analogy would have predicted.

The analogy also explains the industry's shape. The runtime layer — not the model — is where durable engineering value accumulates, exactly as the OS mattered more than the instruction set architecture. Models are swappable CPUs; the scheduler, memory manager, and driver ecosystem are where switching costs and reliability actually live.

The concession is the place the analogy breaks, and it breaks in one precise spot: the kernel is stochastic. A real OS never misreads its own page table; an LLM misreads its context routinely, confabulates syscalls that don't exist, and returns different outputs for identical inputs. No operating system ever needed an eval suite, a verifier model, or a hallucination taxonomy, because determinism was the one property the whole edifice assumed. So the analogy is a design guide for the resource-management half of the problem and silent on the verification half — push it into "the kernel is trustworthy" territory and it will hurt you. The machinery that fills that gap — evals, verifiers, structured-output validation — has no OS ancestor. It's genuinely new.

Used within its limits, though, this is the rare analogy that generates architecture instead of decorating it. When someone asks me why an agent runtime needs budgets, isolation, audit, and paging policy, the answer fits in a sentence: because it's an operating system, and we already know what happens to operating systems that skip those.