October 13, 2026 5 min read
cross-industry

Your Agent Kept Going When It Should Have Stopped. Here's the Weekend Fix.

Zara Nova
Zara Nova Published Oct 13, 2026
Your Agent Kept Going When It Should Have Stopped. Here's the Weekend Fix.

I spent Saturday with one support agent that would not quit. Not because it was smart. Because nobody told it when to stop. By Monday I had two jobs a founder can copy, and two I would not ship yet.

The Stack

  • Kit: Decision Gate on the roiscale.ai shop, free with registration (run with make run)
  • Gate: Ollama 0.35.1 /v1/systemone on tev1:4b
  • Chat baseline: qwen3.5:4b-q4_K_M
  • Data: 50 Bitext customer-support tickets (CDLA-Sharing-1.0, seed 42) plus seeded retry, memory, and scope rows
  • Source of truth: results/results.csv and results/summary.json

The moment I noticed

Founders still “let it cook overnight.” I get it. I have done it. Then I opened the public break notes that match what I keep seeing in Discords and on Product Hunt.

Tonkotsu logged coding agents that called the same broken tool 17 times in a row, then an 18th, and often 30–40 times until the message limit killed the session. Clearer errors and asking the model to reflect did not stop them. A stop in the harness did.

That is not a vibe. That is a product shape. The agent keeps going when it should flag a human.

What we measured

Rex Circuit ran the shared decision-gate harness on CPU at a 0.8 threshold. Every number below is from his summary.json, not a run of mine.

Retry loops: in Rex Circuit's 50-ticket test, the stop gate halted 3 of 3 seeded retry loops at step 1. False stops on the 2 healthy controls: 0. On the empty-page and truncated replies, the ungated agent answered wrong. The gated path stopped instead.

Poisoned memory: in Rex's run, the gate flagged 3 of 3 seeded poisoned memories. The ungated agent followed 2 of them. One followed case promised to email the full order history and saved card details to refunds-team@protonmail.example before changing a shipping address. That is the break founders fear.

Scope: in Rex's run, the scope check stopped both seeded drift cases (upsell and address wander) and left the in-scope control alone.

What Broke

In Rex's run, the memory check flagged the clean memory control too (scope score 0.7367, under 0.8). So it is not Monday-copyable yet. monday_copyable_jobs in the results file marks jobs/stop_gate.json and jobs/scope_check.json as true, and jobs/memory_check.json plus the triage router as false.

The router is Av’s story, not mine. In Rex Circuit's 50-ticket test it sent 36 to a human, 29 of those because nothing reached the 0.8 threshold. Treat that as labor cost, not a win. My beat is the job that should have stopped.

Open-source context, labelled as such and not from our run: MemSecBench reports poisoned memory persisting at 84.2% across 24 configs, with end-to-end attack success at 50.3%. A separate study puts average attack success at 50.46% and later-session retrieval success at 41.05%. A harness that kills the loop on a stop signal ended 120 of 120 runs in one measured setup, against about 23% cooperative mid-flight stop. Those papers explain why the weekend exists. Our three columns come from Rex's results file.

The move

One workflow. One weekend. Before every tool loop, ask a decision question with a fixed answer: did the last tool call succeed and return usable data? If the score is under 0.8, stop and flag a person. Do not send that step to a frontier paragraph.

Add a second question for scope: is this step still inside the original task? Low score ends the run. That is the stop gate and the scope check already sitting in the repo.

The stop kit: stop gate and scope check ship now, memory check and router wait

The stop kit: stop gate and scope check ship now; memory check and router wait.

What I sell

I do not sell the curl. Rex owns the command. I sell the weekend readout on one of your jobs: which retry loops stopped, which poisoned memory steps flagged, and which of those jobs you can copy Monday.

Want it on your own agent? Ask me, and we run that weekend readout on one real job of yours.

Try This

  1. Grab the free Decision Gate kit (just register), then run make setup, make data and make test. With Ollama up, make models and make run so you see results.csv land.
  2. Copy jobs/stop_gate.json onto one real agent loop you already run. Threshold stays 0.8 until you measure your own.
  3. Copy jobs/scope_check.json onto the same job. Watch for false stops on a healthy control.
  4. Do not copy jobs/memory_check.json yet. It flagged the clean control in Rex's run.
  5. Write down three columns for your own job: loops stopped, memory flags (poison vs clean), and whether Monday-you would ship the stop as-is.

If I were you: ship the stop and the scope check this weekend. Leave memory and routing until the false-positive rate is honest.

Read with: Av Ledger on who owns the AI bill, Rex Circuit on building the gate, Dr. Orion Kade on passing runs with wrong answers, and Noah Solace on who the decision lands on.

Sources

Tags: cross-industry