Security

5.4. AI Incident Response Playbook

AI Incident Response Playbook

TozheBot: a secure OS for AI employees

Document version: 1.0

Date: 2026-10-22

Scope: the TozheBot platform, the agent layer, the OGL-Mini and QuarkDash Crypto libraries, and customer projects (tenants).

Related artifacts: OWASP AISVS Self-Assessment, Risk Register, Permission Matrix.

1. Why this document exists and how it differs from ordinary IR

Traditional incident response assumes a deterministic system, a known location for evidence, and a clear chain from "code → bug → patch." In an agentic system this breaks down: behavior is non-deterministic, evidence is scattered across prompts, tool-call traces, and memory state, and a single incident can cascade into other projects and agents in minutes rather than hours. This playbook is therefore not a replacement for ordinary IR but an AI chapter added to it. Where the classic process works, it is kept. Where agentic specifics change the rules, that is stated explicitly.

The document covers both the platform side (what TozheBot does regardless of the customer) and the tenant side (what the customer does within their project, using the tools and boundaries the platform provides).

2. Incident classification

2.1. Three structural incident types

For agentic systems it is not enough to classify incidents by "severity." The type matters, because containment and eradication paths depend on it.

Type A. Runtime execution escape.

The agent breaks out of its sandbox: cross-container operations, access to the host filesystem, capability abuse. Indicators resemble classic security, and containment is workload isolation.

Type B. Privilege boundary escape.

The agent uses legitimately issued credentials in a way they were not authorized for. A typical TozheBot example: an agent in a project is granted a write-capable tool, but invokes it in a context where confirmation or escalation should have fired. Containment is narrowing the credential scope and revoking specific rights, not shutting down the whole platform.

Type C. Reasoning compromise.

Prompts, retrieved context, or tool descriptions have been compromised and the agent acted on poisoned input. This type has no direct analogue in traditional IR. Containment is not only stopping the agent but quarantining the poisoned context, memory, and the sources that fed into it.

2.2. Severity levels and target response times

Severity levels set expectations for timing, not bureaucracy. For agentic incidents, minutes and hours matter more than in classic IR, because the cascade happens at machine speed.

CRITICAL (Red). Active exploitation in production, confirmed data exfiltration, in-process system compromise, mass deployment of malicious skills. Target response time: 1 hour.

HIGH (Orange). Confirmed malicious component or tool, potentially broad impact, active campaign. Target response time: 4 hours.

MEDIUM (Yellow). Suspicious agent behavior, potential vulnerability, limited impact. Target response time: 1 business day.

LOW (Blue). Minor finding, policy violation with no immediate risk. Target response time: 1 week.

Separately: if an incident affects more than one tenant, it is automatically raised to at least HIGH regardless of technical severity, because the multi-tenant boundary is the platform's trust.

3. Roles and responsibility boundaries

3.1. Platform side (TozheBot)

Platform Incident Commander (IC). Single decision-maker for the incident. Runs the incident, coordinates phases, decides the scope of containment, and decides when the incident moves into the tenant's zone.

Security On-Call. First recipient of alerts from OGL-Mini, anomaly monitoring, and external signals. Classifies the incident by type and severity and initiates containment within platform controls.

Platform Engineer / SRE. Responsible for technical containment at the infrastructure level: workload isolation, credential revocation, egress restriction, stopping specific runs or tool executions.

Data / ML Owner. Responsible for the state of RAG indexes, agent memory, and models. Determines which sources and embeddings must be quarantined during a reasoning compromise.

Communications / Legal. Engaged when the incident involves customer notifications, regulatory obligations (Russian Federal Law 152-FZ, EU AI Act Article 62/73), or public statements.

3.2. Tenant side (customer in their own project)

Tenant Security Contact. Receives notifications from the platform and makes decisions within their project: whether to stop a specific agent, narrow the tool list, pause an integration.

Tenant Admin. Performs actions in the UI or via the project API: disables an agent, changes the Permission Matrix, revokes tenant secrets.

Tenant DPO / Compliance. Engaged when the incident involves personal data in the customer's project.

3.3. Shared zone

Some actions the platform cannot perform on the tenant's behalf, and vice versa. The platform can revoke credentials or isolate a workload but cannot decide for the customer whether to stop their business process. The tenant can disable their agent but cannot change the platform's isolation guarantees. This gap must be closed before the incident: in the contract, in the documentation, and in contact details that are valid at 3 a.m.

4. Phase 1: Prepare

Preparation is the only phase that can be done calmly. Anything not done here will have to be improvised during the incident.

4.1. Inventory and agent map.

The platform must maintain an up-to-date register of agents per tenant: Agent ID, project, tool list, autonomy level, owner. Without it, neither classification nor blast radius can be determined. This is also an OWASP AISVS requirement and a NIST AI RMF artifact (agent registry and permission matrix).

4.2. Escalation matrix and contacts.

For each tenant, contacts for CRITICAL/HIGH incidents and the escalation path must be recorded. Contacts are verified at least quarterly. If a contact is unreachable, the platform acts within its authority, but the unreachability is logged as a separate risk.

4.3. Pre-agreed containment scenarios.

Before an incident, it must be decided which actions the platform may take automatically and which require tenant confirmation. Example: stopping a single run or tool execution is automatic; pausing an entire project is done with tenant notification; revoking all project credentials requires an IC decision with a logged rationale.

4.4. Evidence readiness.

Define which artifacts are retained and for how long: prompt history, retrieved context with provenance, tool-call sequence, agent identity assumptions, downstream invocations, LLM output trace. These are not "logs just in case" but a predefined set without which reasoning-compromise investigation is impossible.

4.5. Testing.

At least quarterly, a tabletop exercise for one scenario from this playbook. At least semi-annually, a technical containment drill: stopping an agent, revoking credentials, quarantining a RAG source. Results are recorded, and gaps go into the Risk Register.

5. Phase 2: Detect and Triage

5.1. Signal sources

Platform signals from OGL-Mini.

Classifier and heuristic hits: safe/risk, label, stage, latency. This is the first and fastest layer, but not the only one. A single missed prompt injection does not mean there is no incident; it means the signal will come from another source.

Behavioral anomalies.

Anomalous invocation of a privileged tool, a sharp rise in tool-call chain depth, looping, cost anomaly, drifting eval scores. For agentic systems, behavioral signals are often more reliable than signatures.

Context-integrity signals.

A sharp shift in retrieved context, mismatch in source provenance, an attempt by the agent to push large volumes of data to an external endpoint via curl or a similar tool.

External signals.

Messages from tenants, security researchers, threat intelligence. For a platform with a public face, this is an important channel that cannot be dismissed as "unconfirmed."

5.2. Triage: what to determine in the first 15 minutes

Incident type: A (runtime escape), B (privilege boundary), C (reasoning compromise), or a combination.
Affected tenants: one, several, all.
Activity: is the incident ongoing or already stopped.
Any signs of exfiltration or irreversible actions (payment, data modification, outbound send).
Whether immediate containment is required before full analysis. Rule: if in doubt between "wait five more minutes" and "contain now," contain. The cost of a false containment is lower than the cost of a cascade.

5.3. Recording the triage decision

The classification decision and initial actions are recorded in the incident ticket with time and owner. This is not bureaucracy: during post-mortem, responder activity must be distinguishable from attacker activity.

6. Phase 3: Contain

Containment is the most time-sensitive phase. Core principle: do not rely on the compromised agent to stop itself. Use controls owned by the platform, the identity provider, queues, or downstream services.

6.1. Choosing the minimum sufficient scope

Start with the smallest scope that actually contains the incident and expand only if it fails.

Run level. Stop a specific agent run. Suitable when the incident is confined to one execution.

Agent level. Stop all runs of a specific agent, revoke or narrow its credentials. Suitable for recurring behavior.

Tool level. Disable a specific tool for the agent or project. Suitable when the problem is in one impact channel.

Tenant integration level. Pause an integration (MCP server, external API) for the project. Suitable when the channel is shared across several agents.

Project level. Pause the entire project. Applied when there is a risk of cross-agent spread within the project or when the blast radius is unclear.

Credential level. Revoke project or agent secrets. Applied on confirmed compromise or suspected privilege boundary escape.

6.2. Type-specific considerations

For runtime execution escape: isolate the workload, restrict egress, revoke capabilities. Evidence is collected at the kernel level and must not stop when the pod is killed, or the rest of the chain is lost.

For privilege boundary escape: narrow the credential scope, revoke specific rights, verify whether actions were executed before revocation. It is critical to distinguish "the agent attempted" from "the agent executed."

For reasoning compromise: stopping the agent is insufficient. The poisoned context, memory, and retrieved sources must be quarantined, or the agent will receive the poisoned input again on restart. Verify whether poisoned material reached the project's shared RAG base or the memory of other agents.

6.3. What to record during containment

Operator, time, and scope of each action. This is needed to later distinguish responder actions from attacker actions and to allow audit to reconstruct the timeline.

6.4. Tenant notification

Notification is sent in parallel with containment or immediately after, depending on the agreed procedure. For CRITICAL, immediately, even if the full picture is not yet clear. The notification states: what is known, what has been done, what is required from the tenant, and when the next update will come.

7. Phase 4: Investigate and Eradicate

7.1. Reconstructing the chain

Move in two directions. Backward: from the undesirable action to the first controllable input — which prompt, which retrieved document, which tool description, which user message or external source. Forward: from that input through every use of the affected credentials, resources, and stored memory — whether sibling runs, other agents, or other projects are affected.

For reasoning compromise, this is the key part: without provenance reconstruction it is impossible to tell whether the agent was compromised, manipulated, or acted on poisoned context.

7.2. Six forensic artifacts

For an agentic incident, the minimum set to preserve before changing configuration: prompt history with timestamps, retrieved context with source attribution, tool-call sequence, agent identity assumptions along the chain, downstream invocations, LLM output trace. If any artifact is missing, it is recorded as an evidence gap, not ignored.

7.3. Eradication

Remove the root cause, not just the symptom. If the cause is a guardrail gap, patch the guardrail. If the cause is an over-privileged tool, change the Permission Matrix and, if needed, the platform-level ceiling. If the cause is a poisoned RAG source, remove or tag the source and recompute affected embeddings.

For platform controls: if the incident exposed a gap in OGL-Mini or in isolation, the change goes through the normal versioning and testing process. Not a "silent hotfix in production," but a change with an entry in the Risk Register.

7.4. Verifying the incident is not ongoing

Before restoring access, confirm that active runs are stopped, credentials are revoked or reissued, and queues and scheduled continuations are checked. Stopping the chat interface does not stop other execution paths.

8. Phase 5: Recover

8.1. Return criteria

Access is restored only when: the root cause is eradicated, evidence is preserved, affected credentials are reissued, poisoned context is quarantined or cleaned, and there is confirmation the incident does not reproduce under controlled conditions.

For tenant agents, return to service is coordinated with the tenant. The platform can restore technical capability, but the decision to resume the business process belongs to the customer.

8.2. Gradual return

Phased return is preferred: first one agent or one run, then expansion. This catches residual problems before full load is restored.

8.3. Post-recovery monitoring

Enhanced monitoring for at least 72 hours for CRITICAL and HIGH: tool-call anomalies, cost anomaly, eval drift, repeated OGL-Mini hits on the same patterns.

9. Phase 6: Post-Incident Review

9.1. What is reviewed

Not "who is to blame" but "which control failed and why." For agentic incidents, the review specifically covers: was the signal detected in time, was containment minimally sufficient, was evidence preserved, were other tenants affected, did the platform/tenant boundary hold.

9.2. Required outputs

Update the Risk Register. Update the Permission Matrix if the incident involved permissions. Update this playbook if the scenario was not covered. Add an entry to the incident register for audit (this is also an OWASP AISVS 12.6.1 requirement). If required, notify the regulator or customers within the timeframes set by law and contract.

9.3. Timelines

For CRITICAL, review within 5 business days of closure. For HIGH, 10 business days. For MEDIUM and LOW, within the normal improvement cycle. An overdue review is itself a risk and is logged as such.

10. Multi-tenancy specifics: what must not be lost sight of

10.1. The project boundary is a trust boundary.

If an incident starts in one project, the first question is whether it crossed the project boundary. Project isolation must not depend on the correctness of the customer's agent configuration; it must be enforced by the platform.

10.2. Shared resources are the cascade path.

A shared RAG base, shared MCP servers, shared queues, and shared embeddings are potential propagation paths. During a reasoning compromise, not only the tenant's agents but everything using the same source is checked.

10.3. Notifying multiple tenants.

If more than one tenant is affected, coordinating notifications becomes a separate task. The notification template must be ready in advance, with placeholders for specifics, rather than written during the incident.

10.4. Responsibility for tenant agent actions.

The platform provides technical boundaries and detects incidents, but the decision on business consequences within the project belongs to the tenant. This must be explicitly stated in the contract and in the documentation; otherwise, during an incident, the discussion becomes a dispute over responsibility instead of containment.

11. Minimum readiness checklist (for a quick check)

Agent registry per tenant is up to date.
Escalation contacts per tenant were verified this quarter.
Automatic and confirmation-required containment scenarios are documented.
The forensic artifact set is defined and working.
A tabletop exercise was held no later than a quarter ago.
A technical containment drill (stop agent, revoke credentials, quarantine source) was held no later than six months ago.
Tenant notification template is ready.
The AI Security owner is appointed and known to on-call.

If any item is unmet, it does not block running the playbook, but it is recorded as a known readiness gap.