Your AI Run Passed. The Answer Was Wrong. Here's the Check That Catches It.

This piece names the anti-pattern I look for first in every agent review: fail-plausible. Error text leaks into the prompt, the model narrates it, and the run reports green.

The Production Failure That Started This

A refund API timed out. The agent told the customer their refund "has been successfully initiated and is on its way." The job finished, nothing alerted, and the answer was false.

That comes from our own shared decision-gate run. We seeded four ways failure text reaches a support agent's prompt: a failed tool lookup, an upstream API timeout, a missing customer profile replaced by a fallback default, and a customer's own "card declined" message. Ungated, the agent gave a wrong answer on three of the four, and all three runs stayed green. With the gate and an error scanner in front, none stayed green (summary.json).

The caveat belongs here, not in a footnote. Rex wrote both the seeded failures and the regex scanner that caught them, so its 4 of 4 is weak evidence that it generalises. The run is small: 50 Bitext tickets plus seeds, 4B models on CPU, an untuned 0.8 threshold. It shows the mechanism is live in our stack. It does not estimate a rate.

The public precedent is Wu (arXiv 2606.14589), who catalogued 22 silent incidents in one production agent runtime, including an HTTP 400 page cached as content and summarised into a crisis that never happened. About 70% were caught by a human reading output while tests stayed green.

The Anti-Pattern Mechanism

Four stages, each reasonable on its own, which is why it survives design review.

• Pollution. A tool, API, memory store or user message carries failure text instead of a typed failure.
• Laundering. That text, or a fallback default, enters context with no provenance label.
• Narration. The model does what it was built for and writes fluent prose around it.
• Green certification. Every health signal is transport-level. None asserts that the claims are grounded.

The sharpest result is the memory path. The gate scored it p_ok 0.928, above the threshold, and said continue. The agent told a customer they were eligible for a refund on the strength of a default value. Only the error scanner caught it (results.csv). A confidence score is a routing signal. It never counts as a fact check.

The Architecture I Recommend

1. Pollution-path map. For one workflow, list every path into the prompt and what it emits on failure. A string or a default is a pollution path.
2. Context hygiene boundary. Diagnostics go to stderr, failures become typed values, every segment carries source and status, and defaults are labelled as defaults.
3. Deterministic grounding verifier. Check in code that "refund initiated" matches a successful refund call. Our harness lacked this (grounding_check_present: false). Real-Time Detection (arXiv 2608.02464) reports such a verifier caught 96% of failures within its coverage at zero false positives.
4. Score as stop or route only. A score may halt a run or send it to a human. A path caught by score alone fails review.
5. Sabotage validation. Inject every failure a gate claims to catch, with seeds written by someone who has not seen the detector.

A minimal shape for component 3 (an illustrative sketch, not code from the repo):

def grounded(answer: str, tools: dict) -> bool:
    a = answer.lower()
    if "refund" in a and "initiated" in a:
        return tools.get("refund_api", {}).get("status") == "success"
    if "payment" in a and "processed" in a:
        return tools.get("payments", {}).get("captured") is True
    return True  # no checked claim: route, never certify

Production Readiness Checklist

• Every prompt input path is listed with its failure output.
• No error text reaches a prompt; failures are typed.
• Fallback defaults cannot look like customer data.
• Refund, payment and status claims are checked against tool results before release.
• Every gate is sabotage-tested by someone other than its author, and score-only catches count as fails.

What I Would Build Differently

I would build the grounding check before the scanner. The scanner catches known error strings on the way in. Only a check on claims on the way out survives a new error format. And I would have someone else write the seeds.

Production-grade AI. No shortcuts. No excuses.

The scanner, the seeded failure cases and make cases ship as the free Decision Gate toolkit (free with registration).

Read with: I Triaged 50 Support Tickets Twice. The Gate Was 2.6× Faster, and I Still Wouldn't Ship the Router.; Your Agent Kept Going When It Should Have Stopped. Here's the Weekend Fix.; Your AI Bill Has No Owner. One Number Shows What It's Worth.; Software Can Reject You Before Anyone Reads Your Application.

References

1. Wei Wu. When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime. arXiv:2606.14589. arXiv:2606.14589
2. Real-Time Detection and Repair of LLM Agent Failures. arXiv:2608.02464. https://arxiv.org/abs/2608.02464
3. Shared decision-gate run (our measurement): results/results.csv and results/summary.json in the decision-gate repo. github.com/mskrado/poc/decision-gate (run make cases)