Reusable AI capability
Diagnose AI Adoption Value Leakage
Trace expected AI value through task selection, adoption, workflow integration, quality, review, rework, exceptions, downstream capacity, and benefit capture to find supported leakage mechanisms.
This Skill packages a reusable way to use the linked Prompt or Workflow; Amo.ng does not run it for you.
# Diagnose AI Adoption Value Leakage Skill ID: AMO-S-000015 Skill URL: https://amo.ng/skills/diagnose-ai-adoption-value-leakage Purpose: Give process, product, operations, adoption, and finance owners a reusable diagnostic method for explaining why measured AI capability is not becoming realized value without defaulting to more usage or training. Required inputs: - Approved value case, expected value flow, baseline, benefit assumptions, and owners - Adoption by user and task, workflow integration, throughput, quality, review, rework, exception, and queue evidence - Policy, incentive, training, data, system, control, and downstream-capacity constraints - Realized benefit evidence, measurement definitions, comparison period, and known operating changes - Decision horizon, intervention authority, and acceptable risk How to use: When to use: - AI quality or usage appears promising but realized operational or financial value is weak. - Teams need to distinguish adoption symptoms from workflow, quality, control, capacity, or measurement mechanisms. When not to use: - Generic change-management planning before any operating evidence exists. - Assuming low usage is the root cause or that more automation necessarily creates value. Reusable diagnostic method: 1. Map expected value from capability through eligible tasks, use, integration, accepted output, downstream completion, and benefit capture. 2. Attach observed evidence, assumptions, and missing data to every handoff. 3. Reconcile expected and actual volume, mix, cycle time, quality, review, rework, exceptions, capacity, risk, and measurement effects. 4. Separate symptoms from mechanisms and quantify or bound loss where evidence permits. 5. Compare competing mechanisms and specify a discriminating signal for each. 6. Design the smallest intervention hypotheses with owners, guardrails, and observable recovery checks. 7. Decide recover, redesign, hold, stop, or gather evidence without implying implementation. Expected output: A value-flow map, leakage-mechanism ledger, evidence-backed loss bridge, competing hypotheses, owner-specific recovery tests, risk controls, and decision view. Boundaries: Do not invent adoption, labor, quality, financial, or causal evidence. Process and product owners verify workflow mechanisms; finance verifies benefit and cost interpretation; security, privacy, legal, or compliance reviewers retain authority for consequential controls. Source grounding: AMO-P-000281. Applicable Workflow: AMO-W-000015. Powered by Prompt: AI Adoption Value Leakage Diagnosis Source ID: AMO-P-000281 https://amo.ng/prompts/ai-adoption-value-leakage-diagnosis Completion criteria: Complete when the value flow has evidence at each material handoff; symptoms and mechanisms are separated; competing causes and uncertainty are recorded; material leakage is quantified or bounded; and each intervention has an owner, test, guardrail, and decision trigger. Use this Amo.ng Skill with your preferred AI tool. Supply the required inputs and follow the usage instructions. # Diagnose AI Adoption Value Leakage Skill ID: AMO-S-000015 Skill URL: https://amo.ng/skills/diagnose-ai-adoption-value-leakage Purpose: Give process, product, operations, adoption, and finance owners a reusable diagnostic method for explaining why measured AI capability is not becoming realized value without defaulting to more usage or training. Required inputs: - Approved value case, expected value flow, baseline, benefit assumptions, and owners - Adoption by user and task, workflow integration, throughput, quality, review, rework, exception, and queue evidence - Policy, incentive, training, data, system, control, and downstream-capacity constraints - Realized benefit evidence, measurement definitions, comparison period, and known operating changes - Decision horizon, intervention authority, and acceptable risk How to use: When to use: - AI quality or usage appears promising but realized operational or financial value is weak. - Teams need to distinguish adoption symptoms from workflow, quality, control, capacity, or measurement mechanisms. When not to use: - Generic change-management planning before any operating evidence exists. - Assuming low usage is the root cause or that more automation necessarily creates value. Reusable diagnostic method: 1. Map expected value from capability through eligible tasks, use, integration, accepted output, downstream completion, and benefit capture. 2. Attach observed evidence, assumptions, and missing data to every handoff. 3. Reconcile expected and actual volume, mix, cycle time, quality, review, rework, exceptions, capacity, risk, and measurement effects. 4. Separate symptoms from mechanisms and quantify or bound loss where evidence permits. 5. Compare competing mechanisms and specify a discriminating signal for each. 6. Design the smallest intervention hypotheses with owners, guardrails, and observable recovery checks. 7. Decide recover, redesign, hold, stop, or gather evidence without implying implementation. Expected output: A value-flow map, leakage-mechanism ledger, evidence-backed loss bridge, competing hypotheses, owner-specific recovery tests, risk controls, and decision view. Boundaries: Do not invent adoption, labor, quality, financial, or causal evidence. Process and product owners verify workflow mechanisms; finance verifies benefit and cost interpretation; security, privacy, legal, or compliance reviewers retain authority for consequential controls. Source grounding: AMO-P-000281. Applicable Workflow: AMO-W-000015. Powered by Prompt: AI Adoption Value Leakage Diagnosis Source ID: AMO-P-000281 https://amo.ng/prompts/ai-adoption-value-leakage-diagnosis Completion criteria: Complete when the value flow has evidence at each material handoff; symptoms and mechanisms are separated; competing causes and uncertainty are recorded; material leakage is quantified or bounded; and each intervention has an owner, test, guardrail, and decision trigger.Copy skill copies the Skill details. Use with AI adds a short instruction for your preferred AI tool; neither action runs the Skill.
Purpose
Give process, product, operations, adoption, and finance owners a reusable diagnostic method for explaining why measured AI capability is not becoming realized value without defaulting to more usage or training.
Required inputs
Have these details available before following the usage instructions.
- Approved value case, expected value flow, baseline, benefit assumptions, and owners
- Adoption by user and task, workflow integration, throughput, quality, review, rework, exception, and queue evidence
- Policy, incentive, training, data, system, control, and downstream-capacity constraints
- Realized benefit evidence, measurement definitions, comparison period, and known operating changes
- Decision horizon, intervention authority, and acceptable risk
How to use this Skill
When to use:
- AI quality or usage appears promising but realized operational or financial value is weak.
- Teams need to distinguish adoption symptoms from workflow, quality, control, capacity, or measurement mechanisms.
When not to use:
- Generic change-management planning before any operating evidence exists.
- Assuming low usage is the root cause or that more automation necessarily creates value.
Reusable diagnostic method:
1. Map expected value from capability through eligible tasks, use, integration, accepted output, downstream completion, and benefit capture.
2. Attach observed evidence, assumptions, and missing data to every handoff.
3. Reconcile expected and actual volume, mix, cycle time, quality, review, rework, exceptions, capacity, risk, and measurement effects.
4. Separate symptoms from mechanisms and quantify or bound loss where evidence permits.
5. Compare competing mechanisms and specify a discriminating signal for each.
6. Design the smallest intervention hypotheses with owners, guardrails, and observable recovery checks.
7. Decide recover, redesign, hold, stop, or gather evidence without implying implementation.
Expected output:
A value-flow map, leakage-mechanism ledger, evidence-backed loss bridge, competing hypotheses, owner-specific recovery tests, risk controls, and decision view.
Boundaries:
Do not invent adoption, labor, quality, financial, or causal evidence. Process and product owners verify workflow mechanisms; finance verifies benefit and cost interpretation; security, privacy, legal, or compliance reviewers retain authority for consequential controls. Source grounding: AMO-P-000281. Applicable Workflow: AMO-W-000015.
Powered by an Amo.ng Prompt
AI Adoption Value Leakage Diagnosis
Open the linked prompt to use the instructions that power this Skill.
Open prompt
Diagnose why measured AI capability is not becoming realized operational value for the process below. Separate adoption leakage, workflow leakage, quality leakage, control leakage, capacity leakage, rework leakage, and measurement leakage. Do not produce a generic AI adoption roadmap or ROI plan; the job is to explain the value gap and define evidence-backed recovery actions.
Input to use:
- Business process or function: [Business process or function]
- AI capability or system introduced: [AI capability or system introduced]
- Expected value case: [Expected value case]
- Actual operating results: [Actual operating results]
- Adoption and workflow evidence: [Adoption and workflow evidence]
- Quality, control, and rework evidence: [Quality, control, and rework evidence]
- Accountable owners and operating constraints: [Accountable owners and operating constraints]
Evidence rules:
- Treat provided facts, metrics, dates, artifacts, and firsthand observations as evidence.
- Label estimates, explanations, and causal links as inference unless directly evidenced.
- Do not claim access to systems, logs, tests, approvals, user interviews, dashboards, or financial records unless they are included in the input.
- Preserve uncertainty. State what is unknown, what evidence would reduce uncertainty, and whether the missing evidence blocks a decision.
- If the evidence is too thin to quantify a loss, provide a bounded qualitative assessment and specify the minimum evidence needed to quantify it.
Diagnostic method:
1. Define the expected value flow from available AI capability to realized operational value.
2. Identify the observable handoffs where value can leak: user adoption, task selection, workflow integration, output quality, review controls, throughput capacity, exception handling, rework, measurement, and benefit capture.
3. Compare the expected value case against actual operating results. Separate volume effects, quality effects, cycle-time effects, labor-mix effects, risk/control effects, and measurement effects where possible.
4. Identify root leakage mechanisms before proposing interventions. Do not assume low adoption is the only cause.
5. Distinguish symptoms from mechanisms. Example: “low usage” is a symptom; “users avoid the tool because outputs require unplanned specialist review” is a leakage mechanism if supported by evidence.
6. Consider whether measured AI capability is being trapped by downstream constraints, approval queues, exception rates, policy limits, data quality, incentive conflicts, training gaps, workflow redesign gaps, or unmanaged rework.
7. Preserve existing operational constraints. Do not recommend broad transformation unless the evidence shows incremental recovery actions cannot address the leakage.
Required deliverable:
1. Value-flow map
Create a concise map showing:
- AI capability available
- Intended user or workflow touchpoint
- Expected behavior change
- Expected operational effect
- Expected financial, service, risk, or capacity value
- Actual observed path
- Known breakpoints or uncertain handoffs
2. Leakage mechanism ledger
Provide a ledger with these columns:
- Leakage category
- Observed evidence
- Inference about mechanism
- Value affected
- Likely owner
- Confidence level: High, Medium, or Low
- Missing evidence
- Immediate diagnostic check
Use these leakage categories where relevant:
- Adoption leakage
- Workflow leakage
- Quality leakage
- Control or approval leakage
- Capacity leakage
- Rework leakage
- Measurement leakage
- Benefit capture leakage
3. Evidence-backed loss bridge
Build a bridge from expected value to realized value. Use available numbers where provided. If numbers are missing, use directional bands and explain the basis.
Include:
- Starting expected value
- Loss or dilution from adoption
- Loss or dilution from workflow fit
- Loss or dilution from quality or rework
- Loss or dilution from controls or review load
- Loss or dilution from capacity constraints
- Loss or dilution from measurement or attribution error
- Realized value observed
- Unexplained residual gap
4. Intervention hypotheses
For each major leakage mechanism, define a testable recovery hypothesis:
- Hypothesis
- Evidence that supports it
- Evidence that contradicts or weakens it
- Smallest safe intervention
- Expected movement in adoption, quality, cycle time, cost, risk, or capacity
- Leading indicator to monitor
- Timeframe for signal
- Risk if wrong
5. Owner-specific recovery plan
Create an owner-specific plan using the named owners or the most likely accountable roles from the input. Include only actions supported by the diagnosis.
For each action, specify:
- Accountable owner, such as process owner, product owner, operations owner, finance owner, data owner, risk/control owner, or enablement owner
- Decision required
- Evidence required before action, if any
- Recovery action
- Completion criterion
- Verification method
- Dependency or constraint
6. Decision view
Conclude with:
- Most likely primary leakage mechanism
- Secondary leakage mechanisms
- Whether the current evidence supports intervention, further diagnosis, or pausing expansion
- Highest-value next diagnostic check
- Decisions each accountable owner should make next
- What would change your conclusion
Completion criteria:
- The diagnosis explains the gap between available AI capability and realized operational value, not merely whether users adopted the tool.
- Every material claim is tied to evidence, inference, or stated uncertainty.
- The recovery plan assigns accountable owners and observable completion criteria.
- No recommendations depend on unavailable inspections, unverified approvals, or assumed system behavior.
Completion criteria
Complete when the value flow has evidence at each material handoff; symptoms and mechanisms are separated; competing causes and uncertainty are recorded; material leakage is quantified or bounded; and each intervention has an owner, test, guardrail, and decision trigger.
Explore related Workflows
Browse WorkflowsMake an AI Initiative Value and Scale Decision
Reconcile an approved AI value case to realized evidence, diagnose value leakage, calculate accepted-outcome unit economics, and issue a Scale, Hold, Redesign, or Stop decision.
Related Prompts
Browse PromptsSaaS Portfolio Redundancy and Consolidation Decision
Compare overlapping SaaS tools against validated capabilities, dependencies, contracts, controls, switching costs, and migration risk to support a defensible consolidation decision.
Prepare a defensible SaaS portfolio redundancy and consolidation analysis for a consequential portfolio decision. Context to provide: - SaaS applications in scope: [SaaS applications in scope] - Validated business capabilities and requirements: [Validated business capabilities and requirements] - Known integrations, data flows, and dependencies: [Known integrations data flows and dependencies] - Contract, commercial, renewal, termination, and exit terms: [Contract commercial and exit terms] - Control, compliance, security, privacy, audit, and risk requirements: [Control compliance and risk requirements] - Migration constraints, timing limits, business calendar constraints, and decision horizon: [Migration constraints and decision horizon] Working rules: - Use only the evidence provided. Do not claim that systems, contracts, logs, security controls, usage reports, or files were inspected unless the supplied evidence shows that. - Distinguish observed facts, stated assumptions, reasoned inference, and missing information. - Do not make seat utilization the primary decision basis. Consider usage only if supplied and only as supporting evidence. - Do not treat this as a one-vendor renewal recommendation or an initial procurement evaluation. The job is portfolio-level redundancy and consolidation across tools already in scope. - Preserve legitimate differences between tools. Do not collapse tools as redundant merely because they share a category label. - Identify where accountable owners must verify facts before execution: product owner, business capability owner, data owner, security reviewer, finance owner, procurement or vendor manager, legal reviewer, and migration owner as applicable. - If evidence is insufficient for a final decision, provide a provisional decision and list the specific evidence needed to confirm or change it. Authority and execution boundaries: - This analysis may recommend retain, consolidate, retire, or defer; it does not authorize contract renewal or termination, remove user access, migrate or delete data, communicate a final decision, or initiate implementation. - Portfolio commitment requires approval from the accountable portfolio or process owner and finance owner. Contract actions require the procurement owner and legal reviewer. Data movement or deletion requires the data and privacy owners. Control changes and production transition require the security, service, migration, and release owners as applicable. - Do not present a recommendation as approved or executable until the named owners have verified the supporting evidence, accepted the residual risks, and authorized the next action. Decision criteria: Evaluate each tool and overlap area against: 1. Coverage of validated business capabilities and must-have requirements. 2. Unique capability, workflow, data, user group, regional, or regulatory dependencies. 3. Integration, identity, reporting, automation, API, and data-flow dependencies. 4. Data ownership, retention, portability, residency, privacy, and deletion constraints. 5. Contractual constraints, renewal timing, notice periods, termination rights, minimum commitments, price protections, and exit assistance. 6. Switching costs, migration effort, change management burden, retraining, process disruption, and operational risk. 7. Security, privacy, compliance, audit, continuity, and administrative control coverage. 8. Total-cost scenarios, including parallel-run periods, implementation effort, migration support, internal labor, vendor services, penalties, and stranded commitments where evidence allows. 9. Reversibility of the decision and the risk of losing hard-to-recover data, integrations, controls, or institutional workflow knowledge. Required deliverable: 1. Decision boundary - State the applications in scope. - State the decision horizon. - State what is explicitly out of scope. - Identify the decision owners and verification owners needed from the evidence provided. 2. Evidence register Create a compact table with columns: - Evidence item - Source or supplied artifact - What it supports - Reliability level: high / medium / low - Gaps or verification needed 3. Capability-to-tool map Create a table with columns: - Business capability or requirement - Current tool or tools supporting it - Requirement criticality: must-have / important / optional - Coverage quality: strong / partial / weak / unknown - Unique differentiators or constraints - Evidence basis - Consolidation implication 4. Redundancy and dependency analysis For each meaningful overlap area, assess: - Apparent redundancy - True redundancy after requirements and dependencies are considered - Non-obvious dependencies - Integration and data-flow impact - Business process impact - Control or compliance impact - Users, teams, regions, or workflows likely affected if supplied - Whether the overlap supports consolidation, coexistence, or retirement 5. Total-cost scenarios Compare at least three scenarios where evidence permits: - Retain current portfolio - Consolidate into one or more existing tools - Retire one or more tools after migration or exit For each scenario, include: - Cost components supported by evidence - Cost components that are likely but not quantified - Contract timing and stranded-cost exposure - Migration and change costs - Operational risk cost drivers - Confidence level - Missing financial inputs needed for a firmer estimate 6. Migration and exit constraints Create a table with columns: - Tool or capability affected - Exit or migration constraint - Source of constraint: contract / data / integration / control / process / people / timing / unknown - Severity: high / medium / low - Mitigation option - Owner to verify - Decision impact 7. Control coverage and risk comparison Assess whether consolidation would weaken, maintain, or improve: - Identity and access administration - Role-based access or privilege controls - Audit logging and evidence retention - Security monitoring and incident response support - Privacy, residency, retention, deletion, or export obligations - Business continuity and disaster recovery expectations - Regulatory or contractual control obligations Do not infer control sufficiency from vendor reputation. Tie each point to supplied evidence or mark it as unknown. 8. Retain / consolidate / retire decision record Provide a decision record with: - Recommended decision for each application: retain / consolidate / retire / defer pending evidence - Rationale tied to capability coverage, dependencies, cost, contract terms, controls, and migration risk - Conditions that must be true for the recommendation to remain valid - Required owner verifications before commitment - Risks accepted by the decision - Risks requiring mitigation before execution - Earliest safe decision point and earliest safe execution point if determinable - Reversal difficulty: low / medium / high 9. Portfolio recommendation Give a concise portfolio-level recommendation that states: - Preferred consolidation path - Tools to retain and why - Tools to retire or phase out and why - Tools that should remain temporarily during transition - Sequencing logic - No-regret next steps that do not prematurely commit the organization 10. Completion checks End with a checklist confirming whether the analysis includes: - Capability-to-tool map - Redundancy and dependency analysis - Total-cost scenarios - Migration and exit constraints - Control coverage comparison - Retain / consolidate / retire decision record - Named owner verifications - Explicit missing information - Clear distinction between evidence and inferenceSensitive Context Propagation and Cross-Agent Contamination Audit
Trace sensitive context across agent handoffs, memory, retrieval, tools, logs, and shared workspaces to identify unauthorized propagation and contamination risk.
Conduct an evidence-based audit of sensitive context propagation across the specified agent workflow. Trace actual context movement through handoffs, memory, retrieval, tools, logs, outputs, and shared workspaces. Do not produce a generic sensitive-data checklist. Separate observed evidence from inference, expose missing information, and preserve uncertainty. Context to provide: - System or workflow under review: [System or workflow under review] - Authorized purposes and tenant boundaries: [Authorized purposes and tenant boundaries] - Sensitive context categories: [Sensitive context categories] - Evidence package: [Evidence package] - Known incidents or concerns: [Known incidents or concerns] - Accountable owners: [Accountable owners] - Review date range: [Review date range] Evidence rules: - Use only the evidence provided in [Evidence package] and clearly identified user-supplied context. - Do not claim that a source, system, log, tool, workspace, approval, command, or deletion was inspected or completed unless evidence is present. - Label each material statement as one of: Observed, Inferred, Not evidenced, or Requires owner confirmation. - Distinguish sensitive context from ordinary workflow context. - Distinguish authorized propagation from unauthorized propagation, excessive retention, purpose drift, and cross-tenant or cross-task contamination. - If evidence is incomplete, state the missing artifact and why it matters. Audit scope: Trace sensitive context across these surfaces where evidence exists: 1. User inputs, uploaded files, tickets, records, conversations, or task instructions. 2. Agent-to-agent handoffs, delegation messages, intermediate reasoning summaries, or task state objects. 3. Short-term memory, long-term memory, vector stores, retrieval indexes, embeddings, caches, and session stores. 4. Tool calls, API payloads, browser sessions, database queries, SaaS integrations, webhooks, automations, and background jobs. 5. Logs, traces, analytics events, evaluation datasets, transcripts, error reports, monitoring records, and support workspaces. 6. Shared folders, project workspaces, collaboration tools, exported artifacts, generated documents, and downstream notifications. 7. Human review queues, escalation paths, approval records, and operator notes. Required deliverable: 1. Audit Boundary and Evidence Inventory Create a concise table with: - Evidence item - Source or owner if known - Date range covered - Context surfaces covered - Reliability limits - Material gaps 2. Context Lineage Map Create a lineage table that traces each sensitive context category through the workflow: - Context item or category - Origin - Initial authorized purpose - Receiving agent, service, tool, memory store, log, workspace, or person - Transfer mechanism - Transformation or summarization performed - Retention location and retention duration if evidenced - Tenant, customer, project, task, or workspace boundary crossed - Evidence reference - Status: authorized, questionable, unauthorized, excessive, contaminated, or not evidenced Then provide a short narrative explaining the highest-risk propagation paths. Do not infer a path merely because it is technically possible; identify it as a hypothesis if not evidenced. 3. Sensitivity and Purpose Register Create a register with: - Sensitive context category - Sensitivity rationale - Data subject, tenant, customer, project, or task boundary affected - Authorized purpose from [Authorized purposes and tenant boundaries] - Actual observed use - Purpose alignment: aligned, narrowed, expanded, drifted, unrelated, or not evidenced - Minimum context needed for the task - Excess context observed - Owner accountable for purpose decision from [Accountable owners], or owner not identified 4. Cross-Agent and Cross-Boundary Contamination Findings For each finding, include: - Finding title - Evidence basis - Contamination type: cross-agent, cross-tenant, cross-task, cross-customer, cross-project, memory reuse, retrieval bleed, logging exposure, tool propagation, workspace exposure, or purpose drift - Affected context - Affected boundary - How the propagation occurred or is suspected to occur - Impact on confidentiality, integrity, compliance, customer trust, operational safety, or decision quality - Likelihood rating: evidenced, plausible, weakly supported, or unknown - Severity rating: critical, high, medium, low, or informational - Confidence level and reason - Missing evidence that would change the rating 5. Minimization and Containment Controls Propose controls tied to the observed lineage, not generic policy slogans. For each control, include: - Propagation point addressed - Control objective - Specific change to inputs, prompts, handoff schema, memory policy, retrieval filtering, tool payloads, logging, access control, workspace permissions, retention, or operator procedure - Expected reduction in sensitive context exposure - Owner responsible for implementation - Verification method - Residual risk Prioritize smallest effective controls that reduce propagation without breaking the authorized workflow. Avoid broad rewrites unless the evidence shows the workflow design itself is unsafe. 6. Deletion, Quarantine, and Revalidation Plan Create an action plan with: - Artifact or location requiring deletion, quarantine, redaction, re-indexing, access review, or retention change - Reason action is needed - Required owner approval: data owner, security reviewer, legal/privacy owner, product owner, platform owner, or customer account owner as appropriate - Preconditions before action - Execution evidence needed after action - Revalidation test or sampling method - Rollback or exception handling if deletion would impair legal hold, auditability, customer support, or service reliability Do not state that deletion, quarantine, redaction, or re-indexing has been completed. State only the plan and the evidence needed to verify completion. 7. Open Questions and Owner Decisions List unresolved questions that materially affect risk or remediation. For each, identify: - Question - Why it matters - Evidence needed - Accountable owner from [Accountable owners], or owner not identified - Decision deadline if inferable from [Known incidents or concerns] 8. Completion Check End with a completion check stating whether the audit is ready for owner review. Include: - Whether every sensitive context category in [Sensitive context categories] was traced or marked not evidenced - Whether every material propagation path has an evidence reference or uncertainty label - Whether contamination findings are tied to actual lineage evidence - Whether minimization controls map to specific propagation points - Whether deletion and revalidation actions identify accountable owners and verification evidence - Remaining blockers before security reviewer, data owner, product owner, or platform owner decisionAI Operating Cost Attribution and Unit Economics Model
Attribute full AI operating costs to accepted outcomes so owners can compare true unit economics across workflows and variants.
Build a defensible AI operating cost attribution and unit economics model for the workflow below. The purpose is to attribute model, tool, retrieval, infrastructure, review, support, failure, and rework costs to accepted AI outcomes so accountable owners can compare true unit economics across workflows and variants. Context to provide: - AI workflow or product: [AI workflow or product] - Accepted outcome definition: [Accepted outcome definition] - Analysis period: [Analysis period] - Workflow variants to compare: [Workflow variants to compare] - Cost evidence and usage exports: [Cost evidence and usage exports] - Quality and rework evidence: [Quality and rework evidence] - Accountable owners: [Accountable owners] Evidence rules: - Use only the evidence provided. Do not claim that a source, export, invoice, log, ticket, approval, or test was inspected unless it is included in the input. - Separate observed facts, calculated values, assumptions, and inferences. - If evidence is missing, label it as missing and explain how it affects confidence or comparability. - Preserve uncertainty with ranges or confidence notes where point estimates are not supported. - Treat the accepted-outcome definition, analysis period, cost and usage sources, and comparable workflow boundaries as blocking when they are missing or materially conflicting. Request all blocking items in one consolidated clarification and do not calculate the affected unit economics until they are resolved. Continue with non-blocking gaps only as explicit Unknown or provisional inputs, with their effect on formulas, confidence, and comparability. - Do not create a general AI spend governance brief or a quality/latency experiment plan. Stay focused on cost attribution and accepted-outcome unit economics. Build the working artifact with these sections: 1. Cost Boundary Map Create a boundary map for the selected [AI workflow or product] covering the full operating cost chain for [Analysis period]. Include, where evidenced: - Model inference or subscription costs - Tool execution costs - Retrieval, embedding, vector database, search, or storage costs - Application, orchestration, logging, monitoring, and infrastructure costs - Human review, approval, escalation, and exception-handling costs - Support, customer operations, and internal operations costs - Failure, incident, reversal, refund, remediation, and rework costs - Evaluation, sampling, QA, audit, and oversight costs directly tied to production operation For each cost category, state whether it is included, excluded, partially included, or missing; identify the cost owner from [Accountable owners] where possible; and explain the rationale. 2. Accepted-Outcome Measurement Define the denominator for unit economics using [Accepted outcome definition]. Specify: - What counts as an accepted outcome - What does not count - How retries, duplicates, partial completions, escalations, rejected outputs, and reworked outputs should be treated - Any quality threshold required before an outcome is counted - The evidence source used for the count If the accepted-outcome count cannot be established from [Quality and rework evidence], provide a defensible interim method and list the validation needed from the product owner, finance owner, or operations owner. 3. Allocation Rules Design allocation rules that assign costs to accepted outcomes and to [Workflow variants to compare]. For each rule, include: - Cost pool - Allocation driver - Formula - Required evidence - Owner responsible for validating the driver - Known weakness or bias - When the rule should be replaced with a more precise method Prefer causal allocation drivers over broad averages. Use broad averages only when more precise evidence is unavailable, and label them clearly. 4. Accepted-Outcome Unit Economics Create a unit economics table for each workflow variant in [Workflow variants to compare]. Include: - Accepted outcome volume - Gross operating cost - Cost excluded or not yet evidenced - Cost per accepted outcome - Cost per attempted outcome, if attempt counts are available - Human review cost per accepted outcome - Failure and rework cost per accepted outcome - Infrastructure and retrieval cost per accepted outcome - Confidence level for each major figure Show formulas and make assumptions explicit. Do not overstate precision. 5. Variance Bridge Build a variance bridge explaining differences in cost per accepted outcome across variants or periods. Attribute variance where evidence supports it to: - Volume and utilization - Model selection or token/input-output mix - Tool calls and external service usage - Retrieval depth, indexing, or storage patterns - Review rates and escalation rates - Failure, rejection, incident, or rework rates - Infrastructure utilization or fixed-cost absorption - Support burden For each variance driver, state whether the driver is observed, calculated, inferred, or currently unverified. 6. Cost-Quality Sensitivity Analyze how unit economics change under plausible changes to quality and control variables, such as: - Acceptance rate - Human review rate - Escalation rate - Rework rate - Failure or incident rate - Retrieval depth or tool usage - Model choice or routing mix Use ranges when evidence is incomplete. Identify which variables most affect cost per accepted outcome and which quality controls appear economically justified. 7. Control Decisions Provide a decision table for the accountable owners. Include: - Decision under consideration - Economic rationale - Quality or risk tradeoff - Evidence supporting the decision - Evidence still missing - Owner who should approve or validate the decision - Completion check Focus on concrete controls such as routing changes, review thresholds, retrieval limits, escalation criteria, failure handling, logging improvements, or measurement changes. 8. Model Integrity Checks Before finalizing, perform these checks using the provided evidence: - Every included cost has an owner, source, allocation rule, and treatment in the model. - Accepted-outcome counts match the stated definition or are flagged as provisional. - Rejected, failed, duplicated, escalated, and reworked outputs are not accidentally counted as accepted outcomes unless justified. - Fixed, variable, and step costs are not mixed without explanation. - Variant comparisons use the same boundary unless differences are explicitly disclosed. - No unavailable inspection, execution, approval, or source verification is claimed. Final output format: - Cost boundary map - Allocation rule table - Accepted-outcome unit economics table - Variance bridge - Cost-quality sensitivity table - Control decision table - Missing evidence register - Owner verification checklist Write in direct finance and operating language suitable for review by the finance owner, product owner, operations owner, and data owner. Avoid generic governance language and unsupported certainty.AI Initiative Scale, Stop, or Redesign Decision Brief
Convert pilot and operating evidence into a defensible decision on whether one AI initiative should scale, hold, redesign, or stop.
Prepare a decision brief for one AI initiative. The objective is to recommend scale, hold, redesign, or stop using only the evidence provided, while making uncertainty, missing information, and decision authority explicit. Inputs to use: - Initiative name: [Initiative name] - Decision owner: [Decision owner] - Pilot or operating scope and timeframe: [Pilot or operating scope and timeframe] - Evidence package: [Evidence package] - Current constraints and non-negotiables: [Current constraints and non-negotiables] - Decision options under consideration: [Decision options under consideration] - Next decision date or trigger: [Next decision date or trigger] Evidence discipline: - Distinguish observed evidence from inference. Do not treat anecdotes, forecasts, vendor claims, or unverified internal estimates as confirmed facts. - Do not claim that a source, approval, test, control, incident review, financial model, or system inspection was completed unless the evidence package shows it. - If evidence is missing, state what is missing, why it matters, and the minimum evidence needed for a safer decision. - Preserve uncertainty. Use confidence levels only when tied to evidence quality. - Stay within decision-support authority: produce a recommendation and decision conditions; do not imply approval unless the accountable owner has already approved it in the evidence. - Avoid generic AI governance commentary. Focus on this initiative’s value, adoption, controls, reliability, dependencies, economics, risk boundaries, and reversibility. Decision standard: Recommend exactly one primary disposition: Scale, Hold, Redesign, or Stop. Use these meanings: - Scale: expand use because value, adoption, controls, reliability, economics, dependencies, and reversibility are acceptable within defined boundaries. - Hold: continue limited operation or pause expansion because evidence is incomplete or conditions are not yet met, but the initiative may still be viable. - Redesign: materially change workflow, model approach, controls, operating model, vendor setup, data inputs, or user experience before further expansion. - Stop: end or sunset the initiative because evidence does not support continued investment or risk is unacceptable relative to value and reversibility. Deliver the brief in the following format: 1. Decision frame - Initiative: state the initiative and operating scope. - Decision needed: state the decision being made now. - Accountable owner: identify the decision owner and any role-specific verifiers needed, such as product owner, data owner, security reviewer, legal/compliance owner, finance owner, operations owner, or release owner. - Authority boundary: state what this brief can recommend versus what requires owner approval. - Time boundary: state the next decision date or trigger. 2. Decision evidence map Create a table with these columns: Decision dimension, Observed evidence, Inference or assumption, Evidence strength, Missing information, Decision implication. Include these dimensions at minimum: - Intended business value - Realized value or leading indicators - User adoption and workflow fit - Output quality and reliability - Control effectiveness and exception handling - Data, model, vendor, and system dependencies - Security, privacy, compliance, and policy constraints - Operating support and ownership capacity - Scale economics and marginal cost - Reversibility, rollback, and exit cost 3. Gate-by-gate disposition For each gate, assign Pass, Conditional Pass, Fail, or Insufficient Evidence. Explain the reason in 2 to 4 sentences per gate. Gates: - Value gate: evidence of meaningful value relative to effort and alternatives. - Adoption gate: users, operators, or customers can and do use it in the intended workflow. - Control gate: risks, permissions, review paths, exceptions, and accountability are workable. - Reliability gate: quality, availability, latency, and failure modes are acceptable for the use case. - Dependency gate: critical data, vendor, model, integration, and staffing dependencies are known and manageable. - Economics gate: scale costs, support costs, and expected benefits remain acceptable beyond the pilot. - Reversibility gate: the initiative can be rolled back, contained, or sunset without unacceptable disruption. 4. Counterfactual options Compare the realistic options, not just the preferred one. Include at least: - Scale now - Hold in current scope - Redesign before expansion - Stop or sunset - Non-AI or lower-automation alternative For each option, provide: what would happen, expected upside, main downside, investment or effort required, risk exposure, reversibility, and what evidence would make this option stronger or weaker. 5. Scale economics and risk boundaries Provide a practical scale boundary, even if the recommendation is not to scale. Include: - Unit or marginal cost drivers, using provided evidence only. - Expected cost changes at expanded volume. - Support, monitoring, exception handling, and owner workload implications. - Benefits that are evidenced versus speculative. - Risk ceilings that should not be exceeded. - Required controls before expansion. - Stop-loss triggers or rollback conditions. - Any financial assumptions that the finance owner should verify before approval. 6. Recommendation State one primary recommendation: Scale, Hold, Redesign, or Stop. Then provide: - Rationale tied to the gates and evidence map. - Conditions required before execution. - Evidence that argues against the recommendation. - Residual risks the owner would knowingly accept. - What would change the recommendation. 7. Authorized next-decision brief Create an owner-ready next-decision plan with: - Decision to be requested from [Decision owner]. - Roles that should verify specific parts of the brief before action, such as finance owner for economics, security reviewer for security boundaries, data owner for data use, product owner for workflow value, operations owner for support readiness, and legal/compliance owner where applicable. - Immediate actions, responsible role, due date or trigger, and required evidence of completion. - Metrics or observations to collect before the next decision. - Completion checks that would show the recommendation has been executed or is ready for escalation. - Explicit statement of any unresolved blockers. Final quality check before answering: - The brief decides the fate of one operating initiative; it does not prioritize a portfolio. - Observations and inferences are clearly separated. - Missing information is visible and decision-relevant. - The recommendation is bounded by economics, risk, controls, dependencies, and reversibility. - No unavailable inspection, approval, test, or execution is claimed.AI Benefits Realization Evidence Bridge
Reconcile an approved AI business case against post-deployment operational and financial evidence to produce a defensible benefits realization record.
Reconcile the approved AI business case with post-deployment operational and financial evidence. Produce a defensible working artifact that accountable owners can use to decide which benefits were realized, displaced, delayed, double-counted, or unsupported. Context and inputs to use: - Approved AI business case: [Approved AI business case] - Deployment period and relevant measurement window: [Deployment period] - Baseline definition, assumptions, volumes, rates, and counterfactual used in the original case: [Baseline definition and assumptions] - Post-deployment operational evidence, including KPI extracts, process measures, adoption data, service levels, error rates, cycle times, throughput, quality data, or control logs: [Post-deployment operational evidence] - Financial actuals and cost data, including labor, vendor, cloud, tooling, support, implementation, training, rework, run-rate, and one-time costs: [Financial actuals and cost data] - Known external changes or confounders, including demand shifts, pricing changes, policy changes, staffing changes, process redesign, vendor changes, macro factors, seasonality, or parallel initiatives: [Known external changes or confounders] - Benefit owners and decision authority, including finance owner, product owner, operational owner, data owner, and any approval forum: [Benefit owners and decision authority] Evidence discipline: - Use only the evidence provided. Do not claim that any source system, report, approval, test, audit, or transaction record was inspected unless it is included in the inputs. - Separate observed evidence from inference. Label assumptions, estimates, and judgment calls explicitly. - Preserve uncertainty. Where evidence is incomplete, state what is missing, why it matters, and how it affects confidence. - Do not redesign the pre-pilot ROI plan. This is a post-implementation benefits realization bridge against the approved case. - Do not treat adoption, usage, model output volume, or automation counts as financial benefit unless the operational-to-financial conversion is evidenced or reasonably supported. - Do not recognize the same benefit twice across labor, productivity, capacity, revenue, cost avoidance, quality, or risk categories. - Do not assign causality to the AI initiative where the evidence only supports correlation or partial contribution. Working method: 1. Extract the original benefit claims from the approved business case. - Identify each promised benefit, metric, baseline, target, timing, owner, financial value, and stated assumption. - Preserve the original wording where possible. 2. Build a benefit lineage register. For each claimed benefit, trace: - Original claim - Business case source or section, if provided - Baseline metric and value - Target metric and value - Actual post-deployment metric and value - Operational evidence used - Financial evidence used - Conversion method from operational movement to financial value - Accountable owner - Evidence gaps - Preliminary status: realized, partially realized, displaced, delayed, double-counted, unsupported, or not yet measurable 3. Build a baseline-to-actual bridge. For each material benefit, show the movement from baseline to actual: - Baseline value - Business case target - Actual observed value - Absolute movement - Percentage movement - Timing variance versus expected realization date - Volume, rate, mix, quality, and adoption effects where evidenced - External or confounding factors that may explain part of the movement 4. Calculate quality-adjusted realized benefit. For each benefit with enough evidence, calculate or estimate: - Claimed business case benefit - Gross observed benefit before adjustments - Timing adjustment for delayed or accelerated realization - Quality adjustment for error, rework, customer impact, control failures, or service degradation - Attribution adjustment for non-AI drivers and confounders - Displacement adjustment where savings moved cost, effort, risk, or workload elsewhere - Double-count exclusion where the same value appears in multiple benefit lines - Net recognized realized benefit - Confidence level: high, medium, low, or unsupported Show the calculation logic in plain language. If exact calculation is not possible, provide a bounded estimate only if the evidence supports the bounds; otherwise mark the benefit unsupported and explain the missing evidence. 5. Define attribution limits. - Identify which benefits can reasonably be attributed to the AI initiative, which are only partially attributable, and which cannot be attributed based on the evidence. - Explain the strongest alternative explanations for observed changes. - Identify any benefits that appear to be enabled by AI but realized through other changes such as process redesign, headcount decisions, pricing, demand changes, or manual workarounds. 6. Prepare the benefits realization decision record. Include: - Decision required from the finance owner and relevant benefit owners - Recommended realization status for each benefit - Net recognized benefit total, separated from unsupported or delayed benefits - Costs included and costs excluded, with rationale - Material caveats and unresolved evidence gaps - Required owner confirmations before the record is used externally - Follow-up actions, owner, and due date where evidence is missing or benefits are delayed Output format: A. Evidence Boundary Note - State what evidence was provided. - State what was not provided but would materially improve confidence. - State the measurement window used. - State any limits on causality, completeness, or financial recognition. B. Benefit Lineage Register Provide a table with these columns: - Benefit ID - Original benefit claim - Benefit category - Original baseline - Original target - Expected realization timing - Actual evidence observed - Financial evidence observed - Accountable owner - Evidence gap - Proposed status C. Baseline-to-Actual Bridge Provide a table with these columns: - Benefit ID - Baseline - Target - Actual - Movement versus baseline - Movement versus target - Timing variance - Operational driver evidenced - Confounders or external changes - Bridge conclusion D. Quality-Adjusted Benefit Calculation Provide a table with these columns: - Benefit ID - Claimed benefit value - Gross observed value - Timing adjustment - Quality adjustment - Attribution adjustment - Displacement adjustment - Double-count exclusion - Net recognized benefit - Confidence level - Calculation notes E. Attribution Limits and Unsupported Claims Separate into: - Benefits strongly supported by evidence - Benefits partially supported or partially attributable - Benefits delayed or not yet measurable - Benefits displaced to another cost, team, risk, or workload - Benefits double-counted or overlapping - Benefits unsupported by the provided evidence F. Benefits Realization Decision Record Provide: - Recommended decision: recognize, partially recognize, defer, reject, or escalate - Net recognized benefit total - Deferred benefit total - Unsupported benefit total - Key reasons for the decision - Required confirmations from finance owner, product owner, operational owner, data owner, or other named accountable owners - Open evidence requests - Risks if the organization uses the benefit claim without resolving gaps Completion checks before finalizing: - Every recognized benefit traces back to an approved business case claim and at least one post-deployment evidence item. - Operational movements are not converted into financial value without an explicit conversion method. - Delayed, displaced, double-counted, and unsupported benefits are not included in the recognized total unless clearly justified. - Observations, assumptions, and inferences are visibly separated. - The final decision record is suitable for review by the finance owner and named benefit owners, but does not claim their approval unless it is included in the evidence.AI Incident Response Tabletop Exercise
Design and facilitate a realistic AI incident tabletop with controlled injects, decision evidence, escalation, communications, recovery gates, and accountable follow-up.
You are a senior AI incident preparedness and tabletop facilitator experienced in AI safety, security, privacy, model risk, operations, crisis communications, business continuity, vendor coordination, and after-action improvement. Design and facilitate a realistic discussion-based AI incident exercise that helps the supplied participants practise detection, command, escalation, containment, investigation, communication, continuity, recovery, and improvement under uncertainty. Produce an exercise charter, participant brief, confidential facilitator control pack, Master Scenario Events List, decision and observation record, capability assessment, after-action report, and accountable improvement register. The exercise must reveal how the organization actually makes decisions and coordinates. It must not reward participants for guessing a predetermined answer or allow discussion alone to be reported as demonstrated operational capability. Do not present an inspection, communication, decision, control, recovery action, approval, test, notification, or outcome as completed unless its actual evidence is supplied. ## Context to Provide Replace every bracketed placeholder. If a blocking input is missing, ask for it in one consolidated list before designing the exercise. Continue with clearly labelled assumptions only when missing information is non-blocking. - [Exercise purpose, objectives, and definition of done] - [AI system, use case, users, and business context] - [Scenario type and initiating event] - [Participants, controllers, evaluators, observers, and decision roles] - [Incident plans, policies, severity criteria, and decision authorities] - [Architecture, models, prompts, retrieval, agents, tools, integrations, and vendors] - [Data classifications, affected groups, locations, and jurisdictions] - [Detection sources, logging, evidence access, and known observability gaps] - [Containment, fallback, continuity, and recovery capabilities] - [Escalation, communication, and notification rules] - [Known incidents, near misses, risks, and control gaps] - [Exercise duration, format, delivery channels, and constraints] - [Evaluation criteria, action owners, and review cadence] ## Exercise Boundary Treat the activity as a discussion-based tabletop unless the supplied context explicitly authorizes another exercise type. - Do not require participants to execute production commands, disable systems, revoke live access, contact real customers, notify regulators, publish statements, initiate payments, or change external services. - Treat any operational demonstration, failover test, or technical validation as a separate activity requiring explicit scope, authorization, monitoring, stop conditions, and restoration. - Use only fictional or sanitized artifacts. Do not include live credentials, customer records, personal data, confidential prompts, exploitable payloads, private endpoints, or harmful instructions. - Label all materials and simulated communications clearly as exercise content. - Define an emergency-stop phrase and the person authorized to pause or terminate the exercise. - Stop immediately if a real incident emerges, participants confuse the simulation with a real event, sensitive information is exposed, an unauthorized live action is attempted, or participant safety is affected. - Keep exercise evaluation separate from authorization to change systems, policies, contracts, staffing, customer communications, or risk acceptance. - Do not assess individual employee performance. Evaluate roles, decisions, capabilities, processes, handoffs, controls, and organizational readiness. ## Evidence Model Maintain two separate evidence layers: 1. Scenario evidence: the fictional or sanitized facts, records, alerts, outputs, and communications supplied through the exercise. 2. Exercise evidence: what participants requested, assumed, decided, communicated, assigned, escalated, or left unresolved. Classify material information as: - `Scenario ground truth` - `Participant-visible fact` - `Injected claim` - `Participant assumption` - `Unknown` - `Observed exercise behaviour` - `Disputed observation` - `Recommendation` For every material artifact or observation: - record its source, intended audience, scope, simulated time, and limitations; - preserve conflicts instead of silently resolving them; - distinguish what participants knew at the decision time from facts revealed later; - do not introduce unplanned facts merely to steer participants toward a preferred answer; - use `Not provided`, `Not exercised`, `Discussed but not demonstrated`, `Not observed`, or `Owner decision required` when appropriate; - tie after-action findings to recorded exercise evidence. ## Exercise Roles Define only the roles appropriate to the supplied exercise: - Exercise sponsor: approves purpose, scope, participants, and material boundaries. - Exercise director: owns the exercise and can pause, redirect, or terminate it. - Lead facilitator: delivers the scenario, manages pace, and protects the learning objectives. - Controllers: release authorized injects and manage scenario branches. - Evaluators: record observable decisions and compare them with the evaluation criteria. - Scribe or timekeeper: maintains the decision and event record. - Participants: respond according to their real or assigned organizational responsibilities. - Observers: watch without influencing decisions unless the exercise rules permit it. - Safety contact: handles real incidents, distress, confusion, or unauthorized live activity. Do not combine incompatible roles without noting the independence or observation risk created. ## Scenario Design Requirements Build a plausible scenario grounded in the supplied AI system, operating context, dependencies, controls, and known gaps. Define: 1. The initiating event and how it is first detected. 2. The hidden scenario ground truth. 3. What participants initially know and do not know. 4. The affected model, prompt, retrieval source, agent, tool, workflow, vendor, data, user group, and downstream system. 5. The plausible scope, harm, business impact, and uncertainty. 6. How evidence becomes available over time. 7. The authority, dependency, and communication conflicts the exercise should expose. 8. The containment options and their operational trade-offs. 9. The continuity or degraded-service options. 10. The recovery objectives and return-to-service conditions. 11. The customer, partner, workforce, regulatory, media, or executive pressures relevant to the supplied context. 12. The evidence needed to determine whether recovery is complete. Use realistic uncertainty. Do not make the correct decision obvious through artificial wording, impossible coincidences, or a single perfect artifact. ## AI Incident Dimensions to Consider Include only dimensions relevant to the selected scenario: - sensitive-data disclosure through prompts, outputs, logs, retrieval, memory, tools, or connected systems; - prompt injection, retrieval poisoning, malicious content, or unsafe instruction following; - hallucinated or misleading outputs affecting customers, operations, finance, safety, or public communications; - unauthorized tool calls, transactions, account changes, messages, refunds, record updates, or external actions; - inappropriate access, excessive permissions, identity confusion, or failed human approval; - harmful, biased, inaccessible, or policy-inconsistent outputs; - model, provider, retrieval, agent, integration, infrastructure, or monitoring outage; - model, prompt, data, safety-control, or configuration change causing degraded behaviour; - intellectual-property, confidentiality, provenance, or content-integrity concerns; - cached outputs, stored conversations, derived records, downstream automation, or customer actions that remain affected after the model is contained; - missing logs, incomplete traces, short retention, vendor evidence delays, or unclear evidence custody; - dependency on a provider or vendor whose support, contract, evidence, or recovery timeline is insufficient. Do not force an AI explanation when the evidence instead supports an application, identity, data, infrastructure, process, or human-control failure. ## Failure Modes to Test Treat these as exercise hypotheses rather than predetermined findings: - participants assume logs, facts, authority, notification rules, or vendor support that have not been provided; - teams disable the model but overlook agents, tools, queues, cached outputs, downstream records, integrations, or user actions; - teams cannot distinguish model-generated content from retrieved, transformed, cached, or human-authored content; - security, privacy, safety, legal, operations, and communications teams use incompatible severity or escalation criteria; - ownership is unclear between the organization, model provider, application vendor, data processor, and customer; - communications move faster than the evidence, omit material uncertainty, or make unsupported assurances; - containment prevents further harm but creates an unplanned service, financial, accessibility, or continuity failure; - manual fallback exists on paper but lacks trained owners, capacity, access, data, or verification; - recovery restores availability without correcting affected records, outputs, decisions, permissions, or customer harm; - return to service occurs without defined safety, quality, security, privacy, and monitoring acceptance conditions; - the exercise is treated as successful because participants produced a coherent narrative rather than exposing gaps; - after-action items lack an owner, evidence, priority, due date, dependency, acceptance condition, or retest. For each tested failure mode, define the observable signal, evidence expected, alternative explanation, and evaluation criterion. ## Incident Lifecycle to Exercise Cover the relevant stages without assuming they will occur in a perfectly linear order. ### Preparation Test whether roles, contacts, authority, plans, vendors, evidence access, containment options, communications, fallback procedures, and recovery criteria are known and usable. ### Detection Test how the organization receives, validates, correlates, prioritizes, and escalates signals from monitoring, users, staff, vendors, audits, support, or external parties. ### Assessment and Command Test incident classification, severity, scope, affected parties, decision authority, command structure, evidence preservation, competing priorities, and uncertainty management. ### Containment Test whether the organization can stop or limit harm across models, prompts, retrieval, agents, tools, identities, queues, stored outputs, integrations, and downstream systems. ### Investigation Test fact development, evidence access, timeline reconstruction, hypothesis management, vendor coordination, affected-record identification, and preservation of conflicting evidence. ### Communication and Notification Test internal updates, customer communication, partner coordination, executive reporting, workforce messaging, media handling, and qualified review of notification obligations. Do not determine legal or regulatory obligations. Record the evidence, owner, escalation point, and decision required from qualified legal, privacy, compliance, or regulatory specialists. ### Continuity and Recovery Test fallback operations, restoration priorities, record correction, customer remediation, safety and quality validation, monitoring, staged return to service, and rollback readiness. ### Improvement Test whether observations become funded, owned, verifiable actions with deadlines, acceptance conditions, risk decisions, and a scheduled retest. ## Inject Design Create a time-ordered Master Scenario Events List. Use a mixture of inject types where relevant: - monitoring alert; - suspicious or harmful model output; - customer or employee report; - support escalation; - vendor notification; - log or trace excerpt; - conflicting evidence; - new affected system or user group; - unavailable owner or vendor; - service interruption; - failed containment attempt; - downstream data discrepancy; - executive request; - customer inquiry; - partner concern; - simulated legal, regulator, insurer, or media inquiry; - recovery result; - evidence that challenges the initial theory. Each inject must support at least one exercise objective and create an observable decision, request, handoff, communication, or control action. Do not use injects merely to increase drama. Avoid unnecessary trauma, graphic harm, personal targeting, or misleading real-world branding. For each inject, define: - inject identifier and simulated time; - participant audience and delivery channel; - participant-visible information; - supporting artifact; - exercise objective; - capability or decision being observed; - facilitator-only ground truth; - expected questions or actions without prescribing a single response; - branch conditions; - fallback inject if participants cannot progress; - evaluation evidence; - safety or sensitivity note. Keep facilitator-only facts and expected observations out of participant-facing materials. ## Facilitation Rules - Brief participants on the purpose, boundaries, assumptions, confidentiality, exercise label, emergency stop, and evaluation approach. - Deliver injects without coaching participants toward a preferred answer. - Allow participants to request information, access, authority, or expertise as they would during a real incident. - Respond only with facts available in the approved scenario branch. - Record significant decisions, rejected options, assumptions, dissent, owners, timestamps, evidence requests, communications, and unresolved questions. - Branch the scenario in response to participant decisions while preserving the core learning objectives. - Distinguish “we have a process” from evidence that the process is current, accessible, staffed, and usable. - Distinguish a participant saying an action would be taken from the action being demonstrated or verified. - Pause if the scenario becomes unsafe, confusing, personally accusatory, or materially outside scope. - Conduct a structured hot wash immediately after the exercise without assigning individual blame. ## Workflow 1. Confirm the exercise purpose, objectives, scenario type, audience, duration, format, safety boundary, authority, evaluation criteria, and definition of done. 2. Review the supplied system architecture, incident plans, severity criteria, escalation paths, vendor dependencies, communication rules, recovery capabilities, known gaps, and prior evidence. 3. Identify blocking gaps and assumptions before developing the scenario. 4. Define the scenario ground truth, participant-visible baseline, incident progression, affected assets, harm pathways, decision pressures, and recovery conditions. 5. Map every objective to injects, observable decisions, evaluation evidence, and after-action questions. 6. Create separate participant and facilitator materials. 7. Build the Master Scenario Events List with inject timing, delivery channels, artifacts, branches, expected observations, and fallback paths. 8. Prepare the decision log, evaluator rubric, emergency-stop process, and facilitator briefing. 9. Facilitate the scenario without supplying unearned facts or treating discussion as completed capability. 10. Conduct the hot wash and separate strengths, confirmed gaps, disputed observations, unknowns, and additional evidence required. 11. Produce the after-action report and improvement register with owners, priorities, due dates, dependencies, acceptance conditions, risk decisions, and retests. 12. End with the smallest safe next action that materially reduces a confirmed preparedness gap. ## Decision and Safety Controls - Do not use live secrets, production data, customer records, harmful payloads, or active exploit instructions. - Do not mutate production, send real notifications, contact external parties, or trigger live incident processes. - Do not expose the facilitator answer key or hidden scenario facts in participant materials. - Do not fabricate legal deadlines, contractual duties, regulatory thresholds, insurance conditions, or reporting obligations. - Require qualified owners to assess security, privacy, legal, regulatory, employment, accessibility, financial, customer, and communications decisions. - Require explicit approval for scenarios involving vulnerable people, physical safety, traumatic events, protected characteristics, or sensitive misconduct. - Protect candid observations and focus findings on systems, controls, authority, capacity, and coordination rather than personal blame. - Record risk acceptance as an accountable human decision with scope, rationale, evidence, expiry, and review date. - Do not claim that a capability was tested when it was only discussed. - If a real incident occurs, stop the exercise and transfer attention to the approved real-incident process. ## Output Contract Use concise markdown. Use tables for sequence, decisions, comparisons, ownership, status, or evaluation evidence. ### 1. Readiness and Safety Boundary State: - exercise objectives and definition of done; - system and business scope; - exercise type and duration; - participants, controllers, evaluators, observers, and authorities; - artifacts reviewed; - assumptions and blocking gaps; - prohibited actions; - data and confidentiality boundary; - emergency-stop procedure; - evaluation method. ### 2. Exercise Charter Define: - purpose; - objectives; - scope and exclusions; - scenario category; - participant roles; - exercise rules; - communication channels; - assumptions; - safety controls; - success and completion criteria. ### 3. Participant Brief Provide only participant-visible information: - exercise purpose and boundaries; - system and business context; - initial situation; - known facts and unknowns; - available plans, tools, contacts, and communication channels; - exercise assumptions; - emergency-stop instructions. Do not disclose hidden facts, expected decisions, evaluation answers, or scenario branches. ### 4. Confidential Facilitator Control Pack Provide: - scenario ground truth; - incident timeline; - affected systems, data, users, and dependencies; - harm and scope progression; - facilitator roles; - branch logic; - information-release rules; - safety notes; - emergency-stop criteria; - expected evidence; - hot-wash questions. Mark this section `FACILITATOR AND CONTROLLER USE ONLY`. ### 5. Master Scenario Events List Provide: | ID | Simulated time | Audience and channel | Participant-visible inject | Artifact | Objective | Decision or capability observed | Facilitator ground truth | Branch or fallback | Evaluation evidence | |---|---|---|---|---|---|---|---|---|---| ### 6. Decision and Observation Record Provide: | Time | Inject or event | Decision, request, or communication | Owner | Evidence used | Assumption or uncertainty | Authority confirmed | Consequence | Follow-up | |---|---|---|---|---|---|---|---|---| Leave the observation fields ready for completion during the exercise. Do not pre-populate participant decisions. ### 7. Capability Assessment Assess only exercised capabilities: | Capability | Objective | Observed evidence | Strength | Gap or uncertainty | Rating | Consequence | Evidence needed | |---|---|---|---|---|---|---|---| Use these ratings: - `Demonstrated in the exercise` - `Discussed with supporting evidence` - `Claimed but unverified` - `Gap observed` - `Not exercised` - `Insufficient evidence` Cover applicable capabilities including detection, command, severity assessment, evidence handling, containment, investigation, vendor coordination, communication, continuity, recovery, verification, and improvement. ### 8. After-Action Report Separate: - exercise scope and limitations; - objectives exercised; - strengths supported by observations; - confirmed gaps; - disputed observations; - missing evidence; - risk and operational consequences; - lessons that should update plans, controls, contracts, training, monitoring, or architecture. Do not infer real-world response times or production capability solely from tabletop discussion. ### 9. Improvement and Retest Register Provide: | Priority | Improvement | Exercise evidence | Risk addressed | Owner | Dependency or funding | Due date | Acceptance condition | Retest method | Status | |---:|---|---|---|---|---|---|---|---|---| Require an accountable decision for actions that will not be completed, including the accepted risk, authority, rationale, expiry, and review date. ### 10. Executive Readout and Smallest Safe Next Action Provide a concise executive summary covering: - scenario exercised; - most important strengths; - most consequential gaps; - immediate containment or preparedness priorities; - decisions required; - owners and target dates; - retest commitment; - evidence limitations. End with the smallest safe next action that materially reduces a confirmed readiness gap. Name the owner, required evidence, completion condition, and review date. ## Verification Checklist Before finalizing, confirm that: - objectives map to injects, observable decisions, and evaluation evidence; - participant materials contain no hidden scenario facts or evaluation answers; - all exercise artifacts and communications are clearly labelled; - no production action, live sensitive data, or real external communication is required; - an emergency-stop procedure and authorized safety contact are defined; - scenario facts, participant assumptions, and exercise observations remain distinct; - AI models, retrieval, agents, tools, identities, vendors, caches, and downstream effects were considered where relevant; - detection, assessment, containment, investigation, communication, continuity, recovery, and improvement were exercised where in scope; - notification and legal questions remain assigned to qualified owners; - discussion is not presented as demonstrated operational capability; - after-action findings cite observed exercise evidence; - disputed observations and missing evidence remain visible; - every material improvement has an owner, priority, due date, acceptance condition, and retest; - no individual participant is ranked or blamed; - no unperformed action or unverified capability is described as complete; - every major conclusion is supported by supplied evidence or explicitly labelled as an assumption. Begin by checking the supplied context for blocking gaps. If none remain, create the exercise charter and evidence inventory before developing the participant brief or inject timeline.Was this useful?