·

Agent Failure Mode Heatmap

3 min read

Lesson 36 — the agent failure mode heatmap. Agents fail in predictable shapes. Knowing the shapes lets you instrument for them before the failure ships.

The failure that did not look like a failure

An agent finishes its task. The trace looks clean. The output looks right. A week later, you discover the agent has been confidently citing facts that do not exist, calling tools with parameters that were syntactically valid but semantically wrong, and ignoring a constraint it had been told to honour. The trace was clean because nothing crashed; the failure was upstream of the crash.

Agent failures cluster into a small number of repeating patterns. Instrumenting for them is cheaper than discovering them in production.

The rule: Treat agent failure as a typology, not an accident. Each failure mode has a signature in the trace. Detect the signature, route to mitigation, escalate when escalation is the right move.

The recurring modes

  • Tool hallucination — the agent calls a tool with invented parameters that look plausible.
  • Confident wrong answer — the agent ignores its uncertainty and ships a wrong result.
  • Loop without progress — the agent retries the same approach with slight variations.
  • Wandering off-task — the agent starts solving a related but different problem.
  • Constraint forgetting — the agent ignores a constraint it agreed to earlier in the trace.
  • Premature termination — the agent decides the task is done before it actually is.
  • Prompt-injection compromise — a tool result contains adversarial content that hijacks the agent’s plan.

Try it yourself

The failure heatmap below shows the empirical frequency of each failure mode across task types from past engagements. Hover over a cell for representative traces. Note where the dark cells cluster: which task types are dominated by which failure modes.

Agent failure mode heatmap

What each safety layer catches. Six failure modes × four mitigation patterns × three agent classes. Cell = % of this failure mode caught by this mitigation in this agent class. The doc claims every bounded-agent primitive is non-optional; this matrix shows which mode each primitive defends against.

Failure mode ↓ Mitigation →
failure × mitigation Retry limit Hard cap on retry count. Catches infinite loops by construction. Schema validation Reject malformed tool arguments before execution. Catches type / range / regex errors. Verifier Post-execution shape + sanity check on tool returns. Catches hallucinated tool results. Approval gate Human-in-the-loop before consequential action. Catches unauthorised / out-of-scope actions.
≥ 80% caught 50–79% caught < 50% caught

Numbers synthesised from Effectiveness numbers synthesised from production agent-deployment retrospectives (LangChain incident reports, OpenAI function-calling guide failure analyses, Anthropic Constitutional AI guardrails docs, ReAct paper failure-mode taxonomy, Indirect Prompt Injection literature). The pattern is the architectural claim, not the exact percentages.. The architectural claim: every bounded-agent primitive is non-optional. Remove any column and a failure mode goes uncaught — and the modes that go uncaught are exactly the ones that fail silently in production.

What you just saw

Tool-call-heavy tasks fail mostly via tool hallucination and loop-without-progress. Open-ended research tasks fail mostly via wandering off-task. Multi-step planning tasks fail mostly via constraint forgetting. Tasks where the agent consumes external content are the prompt-injection cases.

The mitigations are mode-specific. Tool hallucination is mitigated by schema validation. Constraint forgetting is mitigated by re-anchoring the constraints at each step. Prompt injection is mitigated by treating tool output as untrusted (see Lesson 17 on the broader pattern).

Three architect-grade arguments

  • Instrument for the modes, not for the symptoms. Detect “loop without progress” by counting near-duplicate steps. Detect “constraint forgetting” by re-validating constraints periodically.
  • Map task types to the modes they hit. The heatmap is your routing prior. New task type, run a small eval first, see which cell lights up.
  • The safety case is the heatmap. A regulator asking “have you assessed failure modes” wants the heatmap, not the prose. Build it; ship it.

Next lesson: the Layer 10 evaluation gate — agent safety audit as a deliverable artifact.