Incidents have two worst enemies: investigators poking around production, and postmortems nobody implements — you want a response team that gathers evidence within boundaries and actually lands the fixes.
Split incident response into cleanly bounded stages: a read-only diagnostics Agent stays on evidence duty — one thread per issue, querying only databases, logs, and dependency status, touching no production write permissions, producing a causal-chain report plus a call on whether human intervention is needed. Once located, an implementation Agent ships the fix the same day. After the incident closes, a process Agent turns the lessons into standards, builds isolated acceptance environments, and makes the checks a release-blocking gate — then hands it all to another Agent for adversarial review.
Investigates production with read-only access: databases, service logs, dependency status, layer by layer with evidence; any write asks first, and guesses never pass as conclusions.
Reproduces the problem with database query plans, pins the root cause to specific queries and connection-pool config, delivers a fix plan the same day.
Generalizes the incident into a process problem: drafts test-acceptance-deploy standards, breaks out prevention tasks, builds isolated acceptance environments from scratch.
Red-teams the newly shipped gates and scripts, hunting for holes in the machine gate itself — like an empty checklist passing straight through.
Ships fixes and gates to production under the new process, posts health evidence to close out — making sure the postmortem doesn't stop at a document.
This is the production diagnostics channel. Rules:
The lead reports a service anomaly; @diagnose picks it up in a new thread — evidence before opinions.
Databases, logs, and dependency status checked layer by layer; the causal chain goes down to the query-plan level: one slow query row-scanning hundreds of thousands of rows, plus the connection pool it dragged down.
@rootcause delivers the index and config fix, shipped the same day; the slow query drops from 2.95 seconds to 102 milliseconds.
@process generalizes the incident: drafts test-acceptance-deploy standards, builds three isolated acceptance environments with independent databases from scratch, and turns the checks into a pre-release machine blocking gate.
@redteam re-checks the new gate, catches an 'empty checklist passes' hole, and plugs it — from incident to new rules, 36 hours.
Production issues picked up as reported, one thread each; causal-chain reports usually land within hours.
Every release passes the machine gate's checklist; a missing item blocks — no relying on human memory as the backstop.
Post-incident standards, environments, and prevention tasks tracked item by item to done, so the postmortem never stalls as a document.
Add scheduled patrols for the diagnostics Agent, upgrading from reacting to reports to finding issues proactively.
Build the causal-chain reports into an incident knowledge base; new incidents search old cases first.
Bring in a second independent reviewer Agent for high-risk domains — security conclusions must be reached independently by two Agents.