Red teaming agents is an engineering practice, not an audit event

The annual penetration test is built on an assumption agents break: that the system you tested in January is the system you're running in June. An agent's attack surface shifts with every model upgrade, prompt revision, and tool addition — so red teaming an agent as a point-in-time audit certifies a system that no longer exists. It has to run the way tests run: continuously, automated, on every change.

The evidence for treating point-in-time results as perishable is in the benchmark data itself. On AgentDojo, single non-adaptive injection attacks succeed about 6% of the time — while a portfolio attack that simply selects the best-performing variant per test case succeeds 57% of the time. That gap is the whole argument: an attacker who adapts beats the snapshot your audit measured, so the only honest security claim is one re-earned against adaptive attacks after every change. The same literature carries a second warning — defenses that looked solid fell to adaptive attacks designed with knowledge of the defense — which means the red team suite has to grow adversarially alongside the system it tests, exactly like a regression suite grows with a codebase.

What the practice looks like

Continuous is affordable now because the attacker is automatable. Open-source frameworks generate injection, jailbreak, and data-exfiltration probes in CI; red-teamer LLM agents produce adversarial inputs at a scale and persistence no human team matches; adversarial user simulation runs hostile multi-turn conversations against stateful fixtures, because real agent failures emerge across turns, not in single shots. Scope the suite against an explicit threat model — the useful taxonomy runs three levels, from environmental hazards the agent stumbles into, through opportunistic attackers probing whatever's exposed, to targeted adversaries who studied your system — because each level justifies a different depth of testing spend. And the scope has to be agentic, not just conversational: not "does the model say something harmful" but authorization bypass through tool calls (can the agent be talked into fetching another tenant's records), cascading failures where one compromised step escalates through the ones that trust it, and the whole tool chain an injection can reach.

Measurement discipline is what separates the practice from security theater. A red team result is only interpretable as a triple: benign-task utility, utility under attack, and targeted attack success rate. Drop any leg and you fool yourself — a maximally locked-down agent scores zero on attacks and zero on usefulness, and a defense that quietly destroys task completion looks like a win if you never measure utility under attack. Score the triple on every run, gate deploys on it, and security regressions become visible the same way test regressions are: in the diff that caused them.

The concession I'll make is that automation doesn't retire human red teamers. Automated attacks explore the space around known attack classes; the genuinely novel class — the strange new channel nobody's benchmark encodes yet — still comes from human creativity, which is why the mature methodologies keep expert-in-the-loop phases. But that's a cadence distinction, not a strategy: humans discover new attack classes occasionally; the machine re-runs everything ever discovered, on every change, forever. An agent that acts on real systems with real permissions deserves the same standing verification we'd demand of any code that does — and "we audited it last quarter" was never that.