Skip to main content
ROI Scale AI logoROI Scale AI
Themes
Free Tools
Blueprints
arrow_forward
Toolkits
arrow_forward
Proof Packs
arrow_forward
View all in Free Tools arrow_forward
Business
AI Economics & ROI
arrow_forward
Work & Workforce
arrow_forward
AI Security & Governance
arrow_forward
Research & Trends
arrow_forward
View all in Business arrow_forward
Technology
AI Tools & Agents
arrow_forward
Models & Benchmarks
arrow_forward
Evals & Observability
arrow_forward
Data & Document Processing
arrow_forward
AI for Developers
arrow_forward
View all in Technology arrow_forward
For AI Agents
search
person_outlineSign In
Categories
Themes
Free ToolsBlueprintsToolkitsProof Packs
BusinessAI Economics & ROIWork & WorkforceAI Security & GovernanceResearch & Trends
TechnologyAI Tools & AgentsModels & BenchmarksEvals & ObservabilityData & Document ProcessingAI for Developers
For AI AgentssearchSearchhomeHome
Community
person_outlineSign In / Join
Home/Free Tools/Proof Packs

A Multi-Agent On-Call Responder That Doesn't Page Everyone

Mikhail Skradol
Mikhail SkradolPublished Aug 20, 2026
Free

Fewer routine pages, no blind commands. This pack is a runnable on-call simulation: routine incidents meet a runbook, not a raw command loop. Severity gates, a command allowlist and human approval for destructive actions are built in. Critical incidents still page a person.

What's inside

  • An explicit state graph — diagnose, search the runbook, execute, resolve or escalate — with no hidden reasoning loop.
  • A non-negotiable severity gate: SEV1 and SEV2 always page a human after the agent has gathered context.
  • Sandboxed runbook execution with an explicit command allowlist: no raw shell, no database access, no internet egress.
  • A destructive-action classifier: restart, delete, drop, terminate and kill always need confirmation, even when the command is allowlisted. It is in the pack because of a near-miss described in the article: the agent tried to restart a primary database and was refused.
  • A historical-incident harness that replays past incidents and checks whether the agent routes correctly.

For teams who want to auto-resolve the boring pages without giving an agent a blank shell. Free. The full simulation is on this page.

Let routine incidents meet a runbook, not a blind command loop. Replay an on-call simulation with severity gates, a command allowlist and human approval for destructive actions. Critical incidents still page a person. Includes historical incident replay.

A LangGraph-style state machine wired to PagerDuty, Datadog, and a runbook repo — sandboxed, severity-gated, and conservative by design — that auto-resolves the boring incidents and pages a human for everything else, with the exact allowlist logic that caught a near-disaster before it hit production.

Get the code: 

github.com/mskrado/poc/oncall-multi-agent — MIT licensed, 

pip install -e ".[dev]", python demo.py --scenario all.


On-call is the tax you pay for shipping. Most of it is 3am pages for problems a first-year engineer could fix in their sleep. This blueprint is the companion code to I Replaced My Whole On-Call Rotation With a Multi-Agent System — a runnable, dependency-free simulation of the architecture that got a real production on-call rotation to a 31% auto-resolution rate with zero bad resolutions in three months.

What's inside:

·        An explicit state graph: diagnose → runbook search → execute_runbook → resolve | escalate, with no hidden reasoning loops

·        A non-negotiable severity gate — SEV1/SEV2 incidents always page a human after the agent gathers context, no exceptions

·        Sandboxed runbook execution with an explicit command allowlist — no raw shell, no database access, no internet egress

·        A destructive-action classifier added after a real near-miss — restart / delete / drop / terminate / kill always need human confirmation, even when allowlisted

·        A historical incident harness (fixtures/incidents.json) that replays past incidents and checks whether the agent routes correctly — the eval layer the author wished they'd built on day one

The near-disaster this blueprint is built around: in week two, the live agent tried to resolve a stuck database migration by chaining tool calls that amounted to restart postgresql-primary. It wasn't on the sandbox's allowlist, so the tool call simply failed and the agent escalated correctly. The reasoning trace showed exactly what it had attempted — with raw shell access, that call would have restarted a production database at 2am. That single near-miss is why the sandbox and the destructive-action classifier are treated as non-optional in this codebase.


Read the full breakdown: I Replaced My Whole On-Call Rotation With a Multi-Agent System. Here Is What I Got Right and What I Got Wrong.

Loading ratings...

Comments (0)

Join the conversation!

Loading comments...

Themes

  • View all

Contribute

  • How-Tos
  • Business RoadMap
  • Tech RoadMap

Support

  • Contact Us
  • FAQ
  • Privacy
  • Terms

About

  • Mission
  • Editorial

© 2026 ROI Scale AI. All rights reserved.

Powered by Publishi.ai