Trust is a design material in AI interfaces
Most AI product teams treat user trust as a score to maximize. That's the wrong objective. A user who trusts an unreliable agent is in more danger than one who doesn't trust it at all — and the failure mode of successful AI products is not skepticism but automation bias: the human stops checking precisely because the system has been right so often. The design goal is calibrated trust — confidence that tracks actual reliability, feature by feature, task by task.
Trust responds to design decisions
Calibration sounds like a training problem, but it's mostly an interface problem. The emerging pattern language for AI UX — Shape of AI's taxonomy is the best current map — is essentially a catalog of trust-calibration instruments: governor patterns (action plans shown before execution, draft modes, cost estimates, verification steps) that keep the human's model of "what will happen next" accurate, and trust-builder patterns (citations, footprints, disclosure of what the system did and why) that let confidence be earned per claim rather than granted wholesale. What they share: each makes the system's competence legible at the moment of decision, instead of asking for faith in the aggregate.
Approval UX is a security property
The sharpest version of this argument comes from twenty years of platform permission design. Windows UAC, Android runtime permissions, iOS purpose strings — the consistent lesson is that the quality of the approval experience determines the security you actually get. UAC's early years are the cautionary tale: prompt too often, with too little context, and users learn to click "yes" reflexively. The dialog remains; the protection evaporates. Android's arc — from install-time bundles to just-in-time requests with rationale, one-time grants, and permanent-denial states — is a slow admission that permission design is a single UX-plus-security problem, not two separable ones.
Agent interfaces are re-running this curve at speed, and permission fatigue arrives much faster when a system asks for approval forty times an hour. Every approval request spends from a finite attention budget. Spend it on reversible trivia and nothing is left for the request that matters — the rubber stamp is the attacker's favorite control, and it's also how a permission boundary quietly stops being one. Ask rarely, with real context (what, why, blast radius, reversibility), and batch the trivial.
Designing for the failure modes
Concretely, calibration means designing against the specific miscalibrations the human-factors literature has already named: automation bias (surface uncertainty selectively, when the system is genuinely less sure, not as a permanent disclaimer that becomes wallpaper); complacency (vary what you ask the human to verify, so oversight stays an active skill); and outcome bias (show the reasoning trail, not just results, so good luck doesn't masquerade as good judgment). And after the agent is wrong — because it will be — trust repair is its own design surface: an agent that names its error, shows what it changed, and demonstrates the fix earns back calibrated confidence; one that silently improves merely teaches the user that the system is unknowable.
None of this is decoration on top of the "real" engineering. In agentic systems the human's decision about when to defer is a system component — arguably the least tested one. Design it with the same rigor as the retry logic.