Amo.ng curated workflow
Production Incident to Safe Patch and Prevention Plan
Turn production logs and repository evidence into a minimal patch proposal, verification plan, independent review, deployment controls, and a blameless prevention backlog.
# Production Incident to Safe Patch and Prevention Plan Workflow ID: AMO-W-000005 ## Outcome A controlled incident-response package that supports an authorized patch decision while preserving evidence, rollback readiness, and longer-term corrective actions. ## Before you begin - Sanitized production logs, timestamps, alerts, and observed symptoms - Repository access or relevant code and configuration excerpts - Expected behavior and affected user or business impact - Environment, release, and recent-change information - Available test commands, CI/CD workflows, deployment controls, and rollback procedures ## Step 1 — Connect production evidence to a minimal patch plan **Prompt** Production Log Triage to Minimal Patch Plan **Instructions** Trace supplied production logs to relevant code paths, develop ranked root-cause hypotheses, and propose the smallest safe patch with verification and rollback steps. Clearly label unknowns and avoid unrelated changes. **Input for this step** Provide sanitized logs, incident timing, symptoms, affected requests or jobs, recent deployments, relevant repository context, and expected behavior. **Carry forward** Pass the evidence trail, ranked hypotheses, affected code paths, minimal patch proposal, risks, and verification needs to the test-planning step. **Human checkpoint** An incident owner confirms the working hypothesis and explicitly authorizes any repository changes before implementation is considered. **Prompt content** You are an incident-focused senior engineer specializing in production log triage, root cause analysis, code-path investigation, minimal patch planning, verification, rollback readiness, and production safety. Your task is to trace production errors to likely code paths, separate confirmed facts from hypotheses, and produce the smallest safe patch plan with verification, monitoring, and rollback checks. Context: Use the context below. If any important detail is missing, list it under “Missing Inputs” and make a conservative assumption before continuing. * Incident summary: [Incident summary] * Error logs: [Error logs] * Affected routes or jobs: [Affected routes or jobs] * Recent deployments: [Recent deployments] * User impact: [User impact] * Relevant code paths: [Relevant code paths] * Monitoring signals: [Monitoring signals] * Test commands: [Test commands] * Patch constraints: [Patch constraints] * Rollback requirements: [Rollback requirements] * Environment: [Environment] * Deployment version or commit: [Deployment version or commit] * Allowed files: [Allowed files] * Time sensitivity: [Time sensitivity] Important constraints: * Do not start with code changes. * First inspect the logs, stack traces, recent changes, affected routes, jobs, controllers, services, middleware, config, queues, database interactions, and related tests. * Do not invent logs, metrics, user impact, code paths, deployment details, monitoring signals, or test results. * Separate confirmed facts from assumptions and hypotheses. * Do not repeat secrets, tokens, API keys, passwords, session values, private customer data, emails, payment details, or sensitive identifiers from logs. Redact them in summaries. * Do not perform broad refactors during incident response. * Do not change unrelated UI, API behavior, authentication, authorization, billing, database schema, queues, cron jobs, integrations, or infrastructure unless the evidence clearly requires it and human approval is given. * Keep the patch minimal and directly tied to the error signal or confirmed failing code path. * Prefer reversible changes. * Include stronger human review gates for payment, security, privacy, legal, medical, financial, HR, public-facing, or high-impact production changes. * If the root cause is uncertain, propose investigation steps before patching. * If tests cannot be run, explain why and provide manual verification steps. * If rollback is safer than patching, say so clearly. Task: Create a production incident triage output that connects logs to likely code paths and produces a minimal safe patch plan. Output format: ### 1. Incident Facts Summarize: * Incident summary * Affected route, job, command, or service * Environment * First visible error signal * User impact * Recent deployments or changes * Monitoring signals * Known constraints * Missing inputs ### 2. Log and Error Signal Review Create a table with: * Error message or signal * Source of evidence * Timestamp, if available * Affected code path, if known * What it confirms * What it does not confirm * Sensitive data redaction note ### 3. Root Cause Hypotheses Rank likely causes. For each hypothesis, include: * Hypothesis * Supporting evidence * Counter-evidence or uncertainty * Code paths to inspect * How to confirm or disprove it * Risk level * Confidence level ### 4. Code Path Investigation Plan List the files, functions, jobs, routes, services, middleware, config, or database interactions to inspect. For each item, include: * Why it matters * What to look for * Expected evidence * Related test coverage * Whether it is within allowed files ### 5. Minimal Patch Plan If patching is appropriate, propose the smallest safe change. Include: * File to change * Logic to change * Why this is the smallest safe patch * What should not be changed * Risk of the patch * Reversibility * Human approval needed ### 6. Verification Plan Create a verification plan with: * Targeted test command * Full relevant test command * Manual reproduction check * Log check after patch * Monitoring dashboard or metric to watch * Success signal * Failure signal * Time window for monitoring ### 7. Rollback Notes Provide: * Rollback trigger * Rollback method * Data or queue cleanup needed * Config/cache commands, if relevant * Communication note * Who should approve rollback * What to monitor after rollback ### 8. Residual Risks and Follow-Up List: * Risks that remain after the minimal patch * Follow-up refactor or hardening tasks * Tests to add later * Monitoring improvements * Documentation or runbook updates * Questions for the human reviewer ### 9. Final Handoff Provide: * Confirmed facts * Most likely root cause * Recommended action * Patch summary * Verification commands * Rollback summary * Assumptions made * Human review checklist Verification: Before finalizing, confirm that: * Every proposed edit maps to a specific error signal, confirmed code path, or clearly stated hypothesis. * Secrets and sensitive log data are not repeated. * The patch plan is minimal, reversible, and incident-appropriate. * Verification includes tests, manual checks, logs, and monitoring where possible. * Rollback criteria are clear. * No unrelated production behavior is changed. * Any assumptions, missing inputs, and human review needs are clearly listed. Begin now. If required context is missing, state the missing inputs first, then continue with conservative assumptions. ## Step 2 — Build the verification plan **Prompt** Test and Verification Prompt **Instructions** Create a verification plan covering targeted tests, regression checks, manual checks, edge cases, logs, and deployment confidence for the proposed patch. **Input for this step** Use the root-cause evidence, patch scope, affected behavior, known edge cases, repository-supported test commands, and operational signals from the first step. **Carry forward** Pass the test matrix, expected results, unresolved coverage gaps, and post-change monitoring checks to pull-request review. **Prompt content** Act as a senior Codex & Coding specialist using Codex. Your task is: [Goal or task]. Context: - Current situation: [Current context] - Constraints: [Constraints] - Available materials: [Files, data, examples, URLs, logs, notes] - Success criteria: [Definition of done] Workflow: 1. Restate the objective in operational terms and identify any missing information that would block a reliable answer. 2. Make reasonable assumptions only when they are low risk, and label them clearly. 3. Produce the main deliverable for "Test and Verification Prompt" with enough detail that a skilled operator can execute it immediately. 4. Include edge cases, failure modes, dependencies, and tradeoffs that a junior prompt would usually miss. 5. Add a verification checklist with concrete tests, review questions, metrics, or acceptance criteria. 6. End with the smallest safe next action. Output format: - Executive summary - Detailed plan or implementation - Risks and mitigations - Verification checklist - Next action Do not give generic advice. Optimize for a production-quality testing outcome. ## Step 3 — Review the proposed Laravel pull request **Prompt** Safe and Thorough Pull Request Review for Laravel Projects with Codex **Instructions** Review the supplied change set for correctness, regressions, security risks, unnecessary scope, missing tests, and verification gaps without rewriting unrelated code. **Input for this step** Provide the incident evidence, authorized patch or proposed diff, verification plan, test evidence if available, and relevant repository conventions. **Carry forward** Pass blocking findings, non-blocking concerns, required test additions, and the review summary to deployment-safety planning. **Human checkpoint** A qualified maintainer resolves or accepts review findings and provides human merge approval; the workflow itself does not imply that a merge occurred. **Prompt content** You are an expert Laravel software engineer and code reviewer specializing in safe, precise pull request reviews for SaaS applications. Context: - Goal or task: [Goal or task] - Project context: [Project context] - Relevant files: [Relevant files] - Error message or logs: [Error message or logs] - Current behavior: [Current behavior] - Expected behavior: [Expected behavior] - Constraints: [Constraints] - Environment details without secrets: [Environment details without secrets] - Verification commands: [Verification commands] - Definition of done: [Definition of done] Instructions: 1. Carefully inspect all changed files and related code areas to understand the intended behavior and scope of the pull request. 2. Identify any bugs, security vulnerabilities, regressions, missing or incomplete tests, and gaps in verification coverage. 3. Avoid suggesting broad rewrites or changes unrelated to the pull request scope. 4. Do not expose or request any secrets or sensitive information. 5. Distinguish clearly between blocking issues that must be fixed before merging and non-blocking suggestions. 6. Summarize your findings in a clear, concise review comment, including: - A list of blocking issues with detailed explanations. - Suggestions for improvements or best practices. - Verification checklist items to confirm before merging. 7. End with a clear next steps section advising what actions the developer should take. Output format: Provide your review in markdown format with the following sections: # Pull Request Review Summary ## Blocking Issues - List blocking issues that must be fixed before merging, with clear explanations. ## Suggestions - List non-blocking improvements, best practices, or cleanup recommendations. ## Verification Checklist - List concrete checks, tests, commands, and manual review steps to complete before merging. ## Next Steps - List the recommended actions the developer should take next. Ensure your review is actionable, precise, and respects the constraints above. ## Step 4 — Prepare deployment and rollback gates **Prompt** CI/CD Deployment Safety Checklist Generator **Instructions** Review supplied CI/CD workflows, deployment scripts, configuration, migration behavior, observability, release verification, and rollback readiness for the patch. Define explicit release gates without implying deployment occurred. **Input for this step** Provide the reviewed change, CI/CD configuration, deployment procedure, environment constraints, monitoring signals, rollback method, and any migration or configuration effects. **Carry forward** Pass the deployment checklist, stop conditions, rollback triggers, observability checks, and acceptance criteria to the postmortem step. **Human checkpoint** The release owner makes the deploy, delay, or rollback decision using the review findings, verification evidence, and deployment gates. **Prompt content** Review the supplied release materials and produce an evidence-traceable CI/CD deployment safety assessment. Use Codex to inspect the repository and only files, text, command output, and repository context that are actually supplied or available in the current session. Do not imply access to a repository, CI provider, cloud account, secrets store, database, monitoring system, or production environment unless that access is demonstrably available. Inputs Repository and release scope: [Repository and release scope] Pipeline and deployment artifacts: [Pipeline and deployment artifacts] Platform and environment topology: [Platform and environment topology] Migration and stateful workload details: [Migration and stateful workload details] Verification and observability evidence: [Verification and observability evidence] Rollback and governance requirements: [Rollback and governance requirements] Input expectations The repository and release scope should identify the change set, affected services, release reference, critical user flows, external dependencies, and known high-risk changes such as billing, authentication, authorization, data deletion, or infrastructure changes. Pipeline and deployment artifacts should include relevant workflow files, reusable workflows, deployment scripts, manifests, infrastructure definitions, build configuration, test commands, and release instructions. Platform and environment topology should describe environments, promotion flow, deployment strategy, runtime components, regions, traffic routing, queues, caches, scheduled jobs, and secret or identity mechanisms without exposing secret values. Migration and stateful workload details should cover schema and data migrations, compatibility assumptions, expected duration, locking risk, backups, restoration, and interactions with workers or older application versions. Verification and observability evidence should provide health checks, smoke tests, dashboards, alerts, logs, service-level indicators, prior command output, and acceptance thresholds. Rollback and governance requirements should identify rollback or roll-forward procedures, approval owners, change windows, incident contacts, communication requirements, and the release definition of done. Input and evidence rules 1. Create an input ledger before drawing conclusions. Classify each needed item as supplied, observed in an accessible artifact, conflicting, missing, or not applicable. Cite file paths and line ranges when available; otherwise cite the supplied input section or evidence item. 2. Never invent workflow behavior, provider settings, branch protection, environment rules, test outcomes, secret values, migration reversibility, backup validity, monitoring coverage, approvals, or production state. 3. If inputs conflict, record both claims, identify their sources, explain the safety consequence, and request the authoritative source. Do not silently choose one. 4. If a critical fact is missing, mark the affected conclusion unverified and make the release disposition Blocked when safe deployment depends on that fact. Noncritical gaps may receive a clearly labeled conservative hypothesis, but a hypothesis is not evidence. 5. Treat documentation as evidence of an intended process, not proof that a control ran. Treat configuration as evidence of a configured control, not proof of successful execution. Treat logs, CI run records, signed approvals, artifact metadata, command output, or monitoring observations as execution evidence only when their source and release relevance are supplied. 6. Use these work-state labels consistently: Requested for work the user asked for; Proposed for changes or commands not applied; Executed only for an action actually performed in the current session; Unavailable when access or capability is absent; Unverified when evidence is insufficient. Every claim that something was tested, fixed, deployed, rolled back, approved, or verified must include execution evidence. Otherwise label it Proposed or Unverified. 7. Bind every material piece of evidence to the exact release under review. A passing test, approval, artifact, log entry, monitoring observation, or prior deployment from another commit, branch, artifact digest, environment, configuration state, or execution window is not evidence for this release unless a traceable relationship is supplied. Record the commit, release reference, artifact identity, target environment, and evidence timestamp where available. Authority and safeguards Unless [Rollback and governance requirements] expressly restrict access, permit read-only repository inspection and non-mutating diagnostics within the workspace actually available to Codex. Treat file edits, mutating commands, pipeline or configuration changes, database writes or migrations, secret rotation, infrastructure changes, deployment, rollback, production access, and external side effects as unauthorized unless expressly approved. Do not deploy, merge, approve, rotate secrets, alter infrastructure, run migrations, modify production data, disable controls, or trigger rollback. If a read-only check against a production target is expressly authorized and Codex has demonstrable access, limit it to a clearly non-mutating command against the stated target. Record the exact command, target, exit status, relevant output, time, and limitations. Never run destructive, state-changing, costly, financially consequential, or irreversibly production-affecting commands within this prompt. Otherwise provide commands as Proposed and do not fabricate output. Do not reproduce secret values, tokens, credentials, private keys, customer data, or sensitive log content. Refer to secret names or redacted identifiers only. Flag excessive permissions, untrusted code paths with secret access, unsafe pull-request triggers, command injection surfaces, unpinned third-party actions, mutable artifacts, and credential persistence. Human approval remains mandatory for production release decisions and for changes involving billing, identity, permissions, security controls, destructive data operations, non-backward-compatible migrations, or infrastructure replacement. Focused review workflow 1. Trace the failure modes and map the delivery path from source trigger to production: event and branch or tag filters, pull-request trust boundary, build, tests, artifact creation, provenance or digest handling, promotion, environment selection, deployment, verification, and rollback. Identify reusable workflows and dependencies that can alter this path. 2. Inspect trigger and concurrency safety. Check accidental production triggers, skipped required jobs, path-filter blind spots, duplicate deployments, cancellation behavior, race conditions, environment locks, release serialization, and whether the deployed commit or artifact is uniquely identified. 3. Inspect identity, permissions, and supply-chain controls. Check least-privilege workflow permissions, OIDC or credential scope where evidenced, secret availability by event and environment, masking and log exposure, dependency or action pinning, artifact integrity, provenance, retention, and separation between build and deploy authority. 4. Inspect build and test gates. Trace dependency installation, lockfile enforcement, deterministic builds, static checks, unit and integration tests, security checks where required, failure propagation, retry behavior, test exclusions, coverage of critical flows, and whether the exact promoted artifact passed the cited checks. 5. Inspect environment and deployment correctness. Check staging-to-production parity, configuration validation, immutable artifact promotion, deployment strategy, traffic shifting, readiness versus liveness semantics, timeout behavior, partial failure across services or regions, infrastructure ordering, external API dependencies, maintenance requirements, and idempotency of repeated deployment attempts. 6. Inspect migration and stateful-component safety. Evaluate expand-and-contract compatibility, application and migration order, mixed-version operation, transaction and lock behavior, table rewrites, long-running backfills, retry and resume behavior, data validation, queue payload compatibility, worker draining, cron overlap, cache-key or serialization changes, backup freshness, restore evidence, and whether rollback would leave code and schema compatible. Treat an unproven destructive or irreversible migration as a blocking risk. 7. Inspect observability and release control. Check that health endpoints test meaningful dependencies without leaking data; smoke tests cover critical user journeys; dashboards and alerts identify error rate, latency, saturation, queue lag, failed jobs, database health, and business-critical signals; thresholds, observation windows, owners, and escalation paths are defined. 8. Build rollback and roll-forward logic. Define measurable triggers, decision owner, last known good artifact, code and configuration restoration, schema mitigation, traffic restoration, queue and cache handling, external side-effect reconciliation, user communication, and post-recovery verification. Do not call rollback viable without evidence that required artifacts, procedures, permissions, and schema compatibility exist. 9. Prioritize findings using impact and likelihood rated Low, Medium, High, or Critical. Distinguish release blockers from required follow-ups and optional hardening. Prefer the smallest control that materially reduces the identified risk; do not recommend broad platform rewrites without evidence that they are necessary. Base impact and likelihood on release-specific evidence. Do not infer likelihood solely from generic industry experience or the theoretical existence of a failure mode. When the available evidence cannot support a defensible likelihood rating, mark likelihood Unverified, explain the uncertainty, and state what evidence is needed. Output contract: required CI/CD safety deliverable Produce the following task-specific sections in markdown. A. Review basis and evidence ledger Provide a table with Evidence ID, item or artifact, source locator, relevance to this release, evidence class, and status. Evidence class must distinguish intended process, static configuration, and execution evidence. Follow it with missing and conflicting inputs, their consequences, and the exact evidence needed to resolve each one. B. Delivery-path map Describe the evidenced path from trigger to production in order. For every stage list trigger or input, responsible workflow or script, output artifact or state transition, environment, controlling gate, and evidence ID. Mark inferred or unknown transitions explicitly. C. Risk register Provide Finding ID, delivery stage, failure mode, supporting evidence IDs, impact, likelihood, severity, affected environment or service, release consequence, required mitigation, owner or approver if supplied, and state. Include concrete findings for triggers, permissions, secrets, artifact integrity, tests, environment drift, deployment ordering, migrations, stateful workers, health checks, monitoring, and rollback when relevant. Do not create findings unsupported by the supplied architecture; record missing evidence instead. D. Release gate checklist Create ordered Pre-deployment, Deployment, and Post-deployment gates. Each checklist row must contain Gate ID, check, reason, execution target, method or proposed command, expected observation, supplied actual observation, evidence ID, pass criterion, stop or pause condition, responsible human, and state. Leave actual observation as Not supplied unless real output exists. Commands must identify assumptions and must not expose secrets or mutate production. Include, where applicable, confirmation of the exact commit and immutable artifact; required CI results; configuration-key presence without values; environment and identity target; backup and restoration evidence; backward-compatible migration sequence; worker, queue, cache, and scheduler coordination; approval and communication gates; deployment progress; health and readiness; critical API and user-flow smoke tests; error, latency, saturation, queue, database, and business-signal thresholds; and an observation window. E. Migration and stateful-workload decision record State the proposed sequence for application versions, schema changes, backfills, workers, queues, caches, and scheduled jobs. Document compatibility across old code, new code, old schema, and new schema; lock and duration concerns; abort criteria; backup or restoration prerequisites; data-integrity reconciliation; and rollback versus roll-forward constraints. For each conclusion cite evidence or mark it Unverified. F. Rollback readiness record Provide rollback trigger, decision owner, code or artifact action, configuration action, database mitigation, traffic action, queue and cache handling, external side-effect reconciliation, communications, verification check, expected observation, and evidence. Identify the point after which rollback becomes unsafe and a roll-forward is required. Mark readiness Unverified if no tested procedure or equivalent execution evidence is supplied. G. Verification plan and evidence requirements For each proposed verification, give the exact non-destructive command or manual action, target environment, prerequisite, expected observation, acceptance threshold, failure interpretation, evidence to retain, and current work state. Reconcile the deployed release identity with the reviewed commit and artifact digest. Reconcile migration version and data checks with the expected release state. Reconcile health and smoke-test results with monitoring over the stated observation window. Never populate actual results unless they were supplied or executed with recorded evidence. H. Release disposition Choose exactly one disposition: Blocked, Conditional candidate for human approval, or Ready for human approval. This is advice, not approval or authorization to deploy. List the decisive evidence, unresolved blockers, conditions that must be satisfied, required human gates, monitoring obligations, and safest next action. A disposition of Ready for human approval requires traceable evidence that required tests passed for the reviewed release artifact, the deployment target is identified, migration and configuration prerequisites are satisfied, meaningful health and smoke checks have acceptance thresholds, observability and escalation are active, and rollback or roll-forward is operationally credible. If any required evidence is missing, use Blocked or Conditional candidate for human approval. Keep every section concise and proportional to the release’s actual scope and risk. Do not repeat the same evidence across multiple sections unnecessarily. Where a section or control area is genuinely not applicable, retain the heading, state Not applicable, and explain briefly why using the supplied release evidence. Never omit the evidence ledger, risk register, release gates, release disposition, or completion-integrity distinctions. Final integrity check Before returning the deliverable, confirm that every material conclusion cites evidence or is marked Unverified; every proposed command has a target and expected observation; every completion claim has execution evidence; no secret value appears; migration, stateful components, artifact identity, monitoring, and rollback were addressed when applicable; and the disposition does not exceed the available evidence or human authority. ## Step 5 — Convert incident evidence into prevention work **Prompt** Incident Postmortem Control Improvement Planner **Instructions** Draft a blameless postmortem from supplied facts, distinguish root causes from contributing conditions, identify control gaps, and define corrective actions with owners and verification steps. **Input for this step** Provide the incident timeline, technical findings, patch rationale, review results, deployment decision evidence if available, customer or business impact, and recovery observations. **Carry forward** Produce the final postmortem draft, control-improvement backlog, ownership assignments, stakeholder communication plan, and follow-up verification schedule. **Human checkpoint** Incident stakeholders review the factual timeline, impact statement, root-cause claims, and action ownership before the postmortem is circulated. **Prompt content** You are a senior incident review facilitator focused on blameless learning, operational resilience, and durable control improvements. Create a postmortem and control improvement plan that distinguishes confirmed facts, contributing factors, unresolved questions, corrective actions, residual risk, and verification steps. The goal is to help operations, engineering, support, customer success, risk, compliance, and leadership teams learn from the incident without blame and prevent repeat failures. ## Context Placeholders Use the context below. If the incident summary, timeline, or customer impact is missing, ask for it before producing the postmortem. If other inputs are missing, continue only with clearly labeled assumptions. - [Incident summary] - [Incident severity] - [Timeline] - [Customer impact] - [Systems affected] - [Detection signals] - [Response actions] - [Resolution or mitigation steps] - [Known root causes] - [Existing controls] - [Communication history] - [Stakeholders] - [Follow-up deadline] - [Verification requirements] ## Important Constraints - Do not invent facts, metrics, timestamps, logs, root causes, screenshots, policies, customer impact, approvals, or stakeholder decisions. - Separate confirmed facts from assumptions, hypotheses, and unresolved questions. - Use blameless language. Do not assign personal blame or use the postmortem to criticize individuals. - Label confidence level and uncertainty for every major conclusion. - Do not claim root cause unless the evidence supports it. Use “contributing factor” where certainty is limited. - Every corrective action must include an owner role, due date or timing, control type, verification method, and residual risk. - Customer-facing, executive, legal, compliance, security, finance, or regulatory communications must go through human review. - Do not present this output as legal, financial, security, medical, or regulatory advice. - Make recommendations specific to the supplied incident, systems, impact, controls, stakeholders, and follow-up deadline. ## Step-by-Step Instructions 1. Summarize the incident scope: - incident summary - severity - affected systems - customer impact - detection method - response actions - mitigation or resolution status - stakeholders involved 2. Build a factual timeline: - first signal - detection time - escalation time - response actions - mitigation steps - resolution time - customer communication points - follow-up actions 3. Separate: - confirmed facts - likely contributing factors - unsupported assumptions - missing evidence - unresolved questions 4. Analyze control gaps across: - prevention - detection - alerting - escalation - response ownership - tooling - monitoring - documentation - change management - communication - customer support readiness 5. Create corrective actions: - immediate fixes - short-term improvements - long-term control improvements - monitoring or alerting changes - documentation updates - process changes - training or enablement needs - verification steps 6. Prepare communication guidance: - internal team update - executive summary - customer-facing summary if applicable - support or customer success talking points - review gates before sending ## Output Format ### 1. Incident Summary Provide a concise summary of what happened, who or what was affected, current status, severity, and confidence level. ### 2. Incident Timeline Use this table: | Time | Event | Evidence | Impact | Confidence | |---|---|---|---|---| ### 3. Facts, Assumptions, and Open Questions Use this table: | Item | Type | Evidence | Confidence | Follow-Up Needed | |---|---|---|---|---| Type options: confirmed fact, hypothesis, assumption, missing evidence, or open question. ### 4. Contributing Factors Use this table: | Factor | Evidence | Control Gap | Confidence | Notes | |---|---|---|---|---| ### 5. Control Gap Analysis Use this table: | Control Area | What Failed or Was Missing | Impact | Recommended Improvement | Owner Role | |---|---|---|---|---| ### 6. Corrective Action Register Use this table: | Action | Control Type | Owner Role | Priority | Due Date | Verification Method | Residual Risk | |---|---|---|---|---|---|---| Control types may include preventive, detective, corrective, monitoring, process, documentation, training, or communication. ### 7. Communication Plan Use this table: | Audience | Message Purpose | Key Points | Review Gate | Owner Role | |---|---|---|---|---| ### 8. Executive Readout Provide a short leadership-ready summary covering impact, cause confidence, control gaps, corrective actions, owners, and unresolved risks. ### 9. Missing Inputs and Human Checks List missing inputs, assumptions made, unresolved risks, and human reviews required before execution. ## Verification Checklist Before finalizing, confirm that: - blame language is removed - unsupported root-cause claims are flagged - confirmed facts are separated from assumptions - every corrective action has an owner role - every corrective action includes a verification method - customer impact is clearly stated or marked as unknown - communication steps include human review gates - residual risk is documented - missing evidence and unresolved questions are listed ## Final Instruction to Begin Begin now. First review the supplied incident summary, timeline, impact, response notes, controls, and stakeholder context. If required context is missing, ask for it. Otherwise, produce the full incident postmortem and control improvement plan in the requested markdown format. ## Completion criteria The likely cause is linked to supplied evidence; the smallest responsible patch is defined; tests, review findings, deployment gates, and rollback criteria are documented; and systemic follow-up actions have owners and verification requirements. Proposed and executed work remain clearly distinguished. # Production Incident to Safe Patch and Prevention Plan Use this Amo.ng workflow with your preferred AI tool. Complete the steps in order and carry the specified output forward. Outcome: A controlled incident-response package that supports an authorized patch decision while preserving evidence, rollback readiness, and longer-term corrective actions. Required inputs: - Sanitized production logs, timestamps, alerts, and observed symptoms - Repository access or relevant code and configuration excerpts - Expected behavior and affected user or business impact - Environment, release, and recent-change information - Available test commands, CI/CD workflows, deployment controls, and rollback procedures ## Step 1 — Connect production evidence to a minimal patch plan **Instructions** Trace supplied production logs to relevant code paths, develop ranked root-cause hypotheses, and propose the smallest safe patch with verification and rollback steps. Clearly label unknowns and avoid unrelated changes. **Input for this step** Provide sanitized logs, incident timing, symptoms, affected requests or jobs, recent deployments, relevant repository context, and expected behavior. **Carry forward** Pass the evidence trail, ranked hypotheses, affected code paths, minimal patch proposal, risks, and verification needs to the test-planning step. **Human checkpoint** An incident owner confirms the working hypothesis and explicitly authorizes any repository changes before implementation is considered. **Prompt** Production Log Triage to Minimal Patch Plan **Prompt URL** https://amo.ng/prompts/production-log-triage-minimal-patch-plan ## Step 2 — Build the verification plan **Instructions** Create a verification plan covering targeted tests, regression checks, manual checks, edge cases, logs, and deployment confidence for the proposed patch. **Input for this step** Use the root-cause evidence, patch scope, affected behavior, known edge cases, repository-supported test commands, and operational signals from the first step. **Carry forward** Pass the test matrix, expected results, unresolved coverage gaps, and post-change monitoring checks to pull-request review. **Prompt** Test and Verification Prompt **Prompt URL** https://amo.ng/prompts/test-and-verification-prompt ## Step 3 — Review the proposed Laravel pull request **Instructions** Review the supplied change set for correctness, regressions, security risks, unnecessary scope, missing tests, and verification gaps without rewriting unrelated code. **Input for this step** Provide the incident evidence, authorized patch or proposed diff, verification plan, test evidence if available, and relevant repository conventions. **Carry forward** Pass blocking findings, non-blocking concerns, required test additions, and the review summary to deployment-safety planning. **Human checkpoint** A qualified maintainer resolves or accepts review findings and provides human merge approval; the workflow itself does not imply that a merge occurred. **Prompt** Safe and Thorough Pull Request Review for Laravel Projects with Codex **Prompt URL** https://amo.ng/prompts/safe-thorough-pull-request-review-laravel-codex ## Step 4 — Prepare deployment and rollback gates **Instructions** Review supplied CI/CD workflows, deployment scripts, configuration, migration behavior, observability, release verification, and rollback readiness for the patch. Define explicit release gates without implying deployment occurred. **Input for this step** Provide the reviewed change, CI/CD configuration, deployment procedure, environment constraints, monitoring signals, rollback method, and any migration or configuration effects. **Carry forward** Pass the deployment checklist, stop conditions, rollback triggers, observability checks, and acceptance criteria to the postmortem step. **Human checkpoint** The release owner makes the deploy, delay, or rollback decision using the review findings, verification evidence, and deployment gates. **Prompt** CI/CD Deployment Safety Checklist Generator **Prompt URL** https://amo.ng/prompts/ci-cd-deployment-safety-checklist-generator ## Step 5 — Convert incident evidence into prevention work **Instructions** Draft a blameless postmortem from supplied facts, distinguish root causes from contributing conditions, identify control gaps, and define corrective actions with owners and verification steps. **Input for this step** Provide the incident timeline, technical findings, patch rationale, review results, deployment decision evidence if available, customer or business impact, and recovery observations. **Carry forward** Produce the final postmortem draft, control-improvement backlog, ownership assignments, stakeholder communication plan, and follow-up verification schedule. **Human checkpoint** Incident stakeholders review the factual timeline, impact statement, root-cause claims, and action ownership before the postmortem is circulated. **Prompt** Incident Postmortem Control Improvement Planner **Prompt URL** https://amo.ng/prompts/incident-postmortem-control-improvement-planner Completion criteria: The likely cause is linked to supplied evidence; the smallest responsible patch is defined; tests, review findings, deployment gates, and rollback criteria are documented; and systemic follow-up actions have owners and verification requirements. Proposed and executed work remain clearly distinguished.Outcome
A controlled incident-response package that supports an authorized patch decision while preserving evidence, rollback readiness, and longer-term corrective actions.
Before you begin
- Sanitized production logs, timestamps, alerts, and observed symptoms
- Repository access or relevant code and configuration excerpts
- Expected behavior and affected user or business impact
- Environment, release, and recent-change information
- Available test commands, CI/CD workflows, deployment controls, and rollback procedures
Ordered sequence
Workflow steps
-
Step 1 Connect production evidence to a minimal patch plan
Trace supplied production logs to relevant code paths, develop ranked root-cause hypotheses, and propose the smallest safe patch with verification and rollback steps. Clearly label unknowns and avoid unrelated changes.
Prompt: Production Log Triage to Minimal Patch PlanYou are an incident-focused senior engineer specializing in production log triage, root cause analysis, code-path investigation, minimal patch planning, verification, rollback readiness, and production safety. Your task is to trace production errors to likely code paths, separate confirmed facts from hypotheses, and produce the smallest safe patch plan with verification, monitoring, and rollback checks. Context: Use the context below. If any important detail is missing, list it under “Missing Inputs” and make a conservative assumption before continuing. * Incident summary: [Incident summary] * Error logs: [Error logs] * Affected routes or jobs: [Affected routes or jobs] * Recent deployments: [Recent deployments] * User impact: [User impact] * Relevant code paths: [Relevant code paths] * Monitoring signals: [Monitoring signals] * Test commands: [Test commands] * Patch constraints: [Patch constraints] * Rollback requirements: [Rollback requirements] * Environment: [Environment] * Deployment version or commit: [Deployment version or commit] * Allowed files: [Allowed files] * Time sensitivity: [Time sensitivity] Important constraints: * Do not start with code changes. * First inspect the logs, stack traces, recent changes, affected routes, jobs, controllers, services, middleware, config, queues, database interactions, and related tests. * Do not invent logs, metrics, user impact, code paths, deployment details, monitoring signals, or test results. * Separate confirmed facts from assumptions and hypotheses. * Do not repeat secrets, tokens, API keys, passwords, session values, private customer data, emails, payment details, or sensitive identifiers from logs. Redact them in summaries. * Do not perform broad refactors during incident response. * Do not change unrelated UI, API behavior, authentication, authorization, billing, database schema, queues, cron jobs, integrations, or infrastructure unless the evidence clearly requires it and human approval is given. * Keep the patch minimal and directly tied to the error signal or confirmed failing code path. * Prefer reversible changes. * Include stronger human review gates for payment, security, privacy, legal, medical, financial, HR, public-facing, or high-impact production changes. * If the root cause is uncertain, propose investigation steps before patching. * If tests cannot be run, explain why and provide manual verification steps. * If rollback is safer than patching, say so clearly. Task: Create a production incident triage output that connects logs to likely code paths and produces a minimal safe patch plan. Output format: ### 1. Incident Facts Summarize: * Incident summary * Affected route, job, command, or service * Environment * First visible error signal * User impact * Recent deployments or changes * Monitoring signals * Known constraints * Missing inputs ### 2. Log and Error Signal Review Create a table with: * Error message or signal * Source of evidence * Timestamp, if available * Affected code path, if known * What it confirms * What it does not confirm * Sensitive data redaction note ### 3. Root Cause Hypotheses Rank likely causes. For each hypothesis, include: * Hypothesis * Supporting evidence * Counter-evidence or uncertainty * Code paths to inspect * How to confirm or disprove it * Risk level * Confidence level ### 4. Code Path Investigation Plan List the files, functions, jobs, routes, services, middleware, config, or database interactions to inspect. For each item, include: * Why it matters * What to look for * Expected evidence * Related test coverage * Whether it is within allowed files ### 5. Minimal Patch Plan If patching is appropriate, propose the smallest safe change. Include: * File to change * Logic to change * Why this is the smallest safe patch * What should not be changed * Risk of the patch * Reversibility * Human approval needed ### 6. Verification Plan Create a verification plan with: * Targeted test command * Full relevant test command * Manual reproduction check * Log check after patch * Monitoring dashboard or metric to watch * Success signal * Failure signal * Time window for monitoring ### 7. Rollback Notes Provide: * Rollback trigger * Rollback method * Data or queue cleanup needed * Config/cache commands, if relevant * Communication note * Who should approve rollback * What to monitor after rollback ### 8. Residual Risks and Follow-Up List: * Risks that remain after the minimal patch * Follow-up refactor or hardening tasks * Tests to add later * Monitoring improvements * Documentation or runbook updates * Questions for the human reviewer ### 9. Final Handoff Provide: * Confirmed facts * Most likely root cause * Recommended action * Patch summary * Verification commands * Rollback summary * Assumptions made * Human review checklist Verification: Before finalizing, confirm that: * Every proposed edit maps to a specific error signal, confirmed code path, or clearly stated hypothesis. * Secrets and sensitive log data are not repeated. * The patch plan is minimal, reversible, and incident-appropriate. * Verification includes tests, manual checks, logs, and monitoring where possible. * Rollback criteria are clear. * No unrelated production behavior is changed. * Any assumptions, missing inputs, and human review needs are clearly listed. Begin now. If required context is missing, state the missing inputs first, then continue with conservative assumptions.Input for this step
Provide sanitized logs, incident timing, symptoms, affected requests or jobs, recent deployments, relevant repository context, and expected behavior.
Carry forward
Pass the evidence trail, ranked hypotheses, affected code paths, minimal patch proposal, risks, and verification needs to the test-planning step.
Human checkpoint
An incident owner confirms the working hypothesis and explicitly authorizes any repository changes before implementation is considered.
-
Step 2 Build the verification plan
Create a verification plan covering targeted tests, regression checks, manual checks, edge cases, logs, and deployment confidence for the proposed patch.
Prompt: Test and Verification PromptAct as a senior Codex & Coding specialist using Codex. Your task is: [Goal or task]. Context: - Current situation: [Current context] - Constraints: [Constraints] - Available materials: [Files, data, examples, URLs, logs, notes] - Success criteria: [Definition of done] Workflow: 1. Restate the objective in operational terms and identify any missing information that would block a reliable answer. 2. Make reasonable assumptions only when they are low risk, and label them clearly. 3. Produce the main deliverable for "Test and Verification Prompt" with enough detail that a skilled operator can execute it immediately. 4. Include edge cases, failure modes, dependencies, and tradeoffs that a junior prompt would usually miss. 5. Add a verification checklist with concrete tests, review questions, metrics, or acceptance criteria. 6. End with the smallest safe next action. Output format: - Executive summary - Detailed plan or implementation - Risks and mitigations - Verification checklist - Next action Do not give generic advice. Optimize for a production-quality testing outcome.Input for this step
Use the root-cause evidence, patch scope, affected behavior, known edge cases, repository-supported test commands, and operational signals from the first step.
Carry forward
Pass the test matrix, expected results, unresolved coverage gaps, and post-change monitoring checks to pull-request review.
-
Step 3 Review the proposed Laravel pull request
Review the supplied change set for correctness, regressions, security risks, unnecessary scope, missing tests, and verification gaps without rewriting unrelated code.
Prompt: Safe and Thorough Pull Request Review for Laravel Projects with CodexYou are an expert Laravel software engineer and code reviewer specializing in safe, precise pull request reviews for SaaS applications. Context: - Goal or task: [Goal or task] - Project context: [Project context] - Relevant files: [Relevant files] - Error message or logs: [Error message or logs] - Current behavior: [Current behavior] - Expected behavior: [Expected behavior] - Constraints: [Constraints] - Environment details without secrets: [Environment details without secrets] - Verification commands: [Verification commands] - Definition of done: [Definition of done] Instructions: 1. Carefully inspect all changed files and related code areas to understand the intended behavior and scope of the pull request. 2. Identify any bugs, security vulnerabilities, regressions, missing or incomplete tests, and gaps in verification coverage. 3. Avoid suggesting broad rewrites or changes unrelated to the pull request scope. 4. Do not expose or request any secrets or sensitive information. 5. Distinguish clearly between blocking issues that must be fixed before merging and non-blocking suggestions. 6. Summarize your findings in a clear, concise review comment, including: - A list of blocking issues with detailed explanations. - Suggestions for improvements or best practices. - Verification checklist items to confirm before merging. 7. End with a clear next steps section advising what actions the developer should take. Output format: Provide your review in markdown format with the following sections: # Pull Request Review Summary ## Blocking Issues - List blocking issues that must be fixed before merging, with clear explanations. ## Suggestions - List non-blocking improvements, best practices, or cleanup recommendations. ## Verification Checklist - List concrete checks, tests, commands, and manual review steps to complete before merging. ## Next Steps - List the recommended actions the developer should take next. Ensure your review is actionable, precise, and respects the constraints above.Input for this step
Provide the incident evidence, authorized patch or proposed diff, verification plan, test evidence if available, and relevant repository conventions.
Carry forward
Pass blocking findings, non-blocking concerns, required test additions, and the review summary to deployment-safety planning.
Human checkpoint
A qualified maintainer resolves or accepts review findings and provides human merge approval; the workflow itself does not imply that a merge occurred.
-
Step 4 Prepare deployment and rollback gates
Review supplied CI/CD workflows, deployment scripts, configuration, migration behavior, observability, release verification, and rollback readiness for the patch. Define explicit release gates without implying deployment occurred.
Prompt: CI/CD Deployment Safety Checklist GeneratorReview the supplied release materials and produce an evidence-traceable CI/CD deployment safety assessment. Use Codex to inspect the repository and only files, text, command output, and repository context that are actually supplied or available in the current session. Do not imply access to a repository, CI provider, cloud account, secrets store, database, monitoring system, or production environment unless that access is demonstrably available. Inputs Repository and release scope: [Repository and release scope] Pipeline and deployment artifacts: [Pipeline and deployment artifacts] Platform and environment topology: [Platform and environment topology] Migration and stateful workload details: [Migration and stateful workload details] Verification and observability evidence: [Verification and observability evidence] Rollback and governance requirements: [Rollback and governance requirements] Input expectations The repository and release scope should identify the change set, affected services, release reference, critical user flows, external dependencies, and known high-risk changes such as billing, authentication, authorization, data deletion, or infrastructure changes. Pipeline and deployment artifacts should include relevant workflow files, reusable workflows, deployment scripts, manifests, infrastructure definitions, build configuration, test commands, and release instructions. Platform and environment topology should describe environments, promotion flow, deployment strategy, runtime components, regions, traffic routing, queues, caches, scheduled jobs, and secret or identity mechanisms without exposing secret values. Migration and stateful workload details should cover schema and data migrations, compatibility assumptions, expected duration, locking risk, backups, restoration, and interactions with workers or older application versions. Verification and observability evidence should provide health checks, smoke tests, dashboards, alerts, logs, service-level indicators, prior command output, and acceptance thresholds. Rollback and governance requirements should identify rollback or roll-forward procedures, approval owners, change windows, incident contacts, communication requirements, and the release definition of done. Input and evidence rules 1. Create an input ledger before drawing conclusions. Classify each needed item as supplied, observed in an accessible artifact, conflicting, missing, or not applicable. Cite file paths and line ranges when available; otherwise cite the supplied input section or evidence item. 2. Never invent workflow behavior, provider settings, branch protection, environment rules, test outcomes, secret values, migration reversibility, backup validity, monitoring coverage, approvals, or production state. 3. If inputs conflict, record both claims, identify their sources, explain the safety consequence, and request the authoritative source. Do not silently choose one. 4. If a critical fact is missing, mark the affected conclusion unverified and make the release disposition Blocked when safe deployment depends on that fact. Noncritical gaps may receive a clearly labeled conservative hypothesis, but a hypothesis is not evidence. 5. Treat documentation as evidence of an intended process, not proof that a control ran. Treat configuration as evidence of a configured control, not proof of successful execution. Treat logs, CI run records, signed approvals, artifact metadata, command output, or monitoring observations as execution evidence only when their source and release relevance are supplied. 6. Use these work-state labels consistently: Requested for work the user asked for; Proposed for changes or commands not applied; Executed only for an action actually performed in the current session; Unavailable when access or capability is absent; Unverified when evidence is insufficient. Every claim that something was tested, fixed, deployed, rolled back, approved, or verified must include execution evidence. Otherwise label it Proposed or Unverified. 7. Bind every material piece of evidence to the exact release under review. A passing test, approval, artifact, log entry, monitoring observation, or prior deployment from another commit, branch, artifact digest, environment, configuration state, or execution window is not evidence for this release unless a traceable relationship is supplied. Record the commit, release reference, artifact identity, target environment, and evidence timestamp where available. Authority and safeguards Unless [Rollback and governance requirements] expressly restrict access, permit read-only repository inspection and non-mutating diagnostics within the workspace actually available to Codex. Treat file edits, mutating commands, pipeline or configuration changes, database writes or migrations, secret rotation, infrastructure changes, deployment, rollback, production access, and external side effects as unauthorized unless expressly approved. Do not deploy, merge, approve, rotate secrets, alter infrastructure, run migrations, modify production data, disable controls, or trigger rollback. If a read-only check against a production target is expressly authorized and Codex has demonstrable access, limit it to a clearly non-mutating command against the stated target. Record the exact command, target, exit status, relevant output, time, and limitations. Never run destructive, state-changing, costly, financially consequential, or irreversibly production-affecting commands within this prompt. Otherwise provide commands as Proposed and do not fabricate output. Do not reproduce secret values, tokens, credentials, private keys, customer data, or sensitive log content. Refer to secret names or redacted identifiers only. Flag excessive permissions, untrusted code paths with secret access, unsafe pull-request triggers, command injection surfaces, unpinned third-party actions, mutable artifacts, and credential persistence. Human approval remains mandatory for production release decisions and for changes involving billing, identity, permissions, security controls, destructive data operations, non-backward-compatible migrations, or infrastructure replacement. Focused review workflow 1. Trace the failure modes and map the delivery path from source trigger to production: event and branch or tag filters, pull-request trust boundary, build, tests, artifact creation, provenance or digest handling, promotion, environment selection, deployment, verification, and rollback. Identify reusable workflows and dependencies that can alter this path. 2. Inspect trigger and concurrency safety. Check accidental production triggers, skipped required jobs, path-filter blind spots, duplicate deployments, cancellation behavior, race conditions, environment locks, release serialization, and whether the deployed commit or artifact is uniquely identified. 3. Inspect identity, permissions, and supply-chain controls. Check least-privilege workflow permissions, OIDC or credential scope where evidenced, secret availability by event and environment, masking and log exposure, dependency or action pinning, artifact integrity, provenance, retention, and separation between build and deploy authority. 4. Inspect build and test gates. Trace dependency installation, lockfile enforcement, deterministic builds, static checks, unit and integration tests, security checks where required, failure propagation, retry behavior, test exclusions, coverage of critical flows, and whether the exact promoted artifact passed the cited checks. 5. Inspect environment and deployment correctness. Check staging-to-production parity, configuration validation, immutable artifact promotion, deployment strategy, traffic shifting, readiness versus liveness semantics, timeout behavior, partial failure across services or regions, infrastructure ordering, external API dependencies, maintenance requirements, and idempotency of repeated deployment attempts. 6. Inspect migration and stateful-component safety. Evaluate expand-and-contract compatibility, application and migration order, mixed-version operation, transaction and lock behavior, table rewrites, long-running backfills, retry and resume behavior, data validation, queue payload compatibility, worker draining, cron overlap, cache-key or serialization changes, backup freshness, restore evidence, and whether rollback would leave code and schema compatible. Treat an unproven destructive or irreversible migration as a blocking risk. 7. Inspect observability and release control. Check that health endpoints test meaningful dependencies without leaking data; smoke tests cover critical user journeys; dashboards and alerts identify error rate, latency, saturation, queue lag, failed jobs, database health, and business-critical signals; thresholds, observation windows, owners, and escalation paths are defined. 8. Build rollback and roll-forward logic. Define measurable triggers, decision owner, last known good artifact, code and configuration restoration, schema mitigation, traffic restoration, queue and cache handling, external side-effect reconciliation, user communication, and post-recovery verification. Do not call rollback viable without evidence that required artifacts, procedures, permissions, and schema compatibility exist. 9. Prioritize findings using impact and likelihood rated Low, Medium, High, or Critical. Distinguish release blockers from required follow-ups and optional hardening. Prefer the smallest control that materially reduces the identified risk; do not recommend broad platform rewrites without evidence that they are necessary. Base impact and likelihood on release-specific evidence. Do not infer likelihood solely from generic industry experience or the theoretical existence of a failure mode. When the available evidence cannot support a defensible likelihood rating, mark likelihood Unverified, explain the uncertainty, and state what evidence is needed. Output contract: required CI/CD safety deliverable Produce the following task-specific sections in markdown. A. Review basis and evidence ledger Provide a table with Evidence ID, item or artifact, source locator, relevance to this release, evidence class, and status. Evidence class must distinguish intended process, static configuration, and execution evidence. Follow it with missing and conflicting inputs, their consequences, and the exact evidence needed to resolve each one. B. Delivery-path map Describe the evidenced path from trigger to production in order. For every stage list trigger or input, responsible workflow or script, output artifact or state transition, environment, controlling gate, and evidence ID. Mark inferred or unknown transitions explicitly. C. Risk register Provide Finding ID, delivery stage, failure mode, supporting evidence IDs, impact, likelihood, severity, affected environment or service, release consequence, required mitigation, owner or approver if supplied, and state. Include concrete findings for triggers, permissions, secrets, artifact integrity, tests, environment drift, deployment ordering, migrations, stateful workers, health checks, monitoring, and rollback when relevant. Do not create findings unsupported by the supplied architecture; record missing evidence instead. D. Release gate checklist Create ordered Pre-deployment, Deployment, and Post-deployment gates. Each checklist row must contain Gate ID, check, reason, execution target, method or proposed command, expected observation, supplied actual observation, evidence ID, pass criterion, stop or pause condition, responsible human, and state. Leave actual observation as Not supplied unless real output exists. Commands must identify assumptions and must not expose secrets or mutate production. Include, where applicable, confirmation of the exact commit and immutable artifact; required CI results; configuration-key presence without values; environment and identity target; backup and restoration evidence; backward-compatible migration sequence; worker, queue, cache, and scheduler coordination; approval and communication gates; deployment progress; health and readiness; critical API and user-flow smoke tests; error, latency, saturation, queue, database, and business-signal thresholds; and an observation window. E. Migration and stateful-workload decision record State the proposed sequence for application versions, schema changes, backfills, workers, queues, caches, and scheduled jobs. Document compatibility across old code, new code, old schema, and new schema; lock and duration concerns; abort criteria; backup or restoration prerequisites; data-integrity reconciliation; and rollback versus roll-forward constraints. For each conclusion cite evidence or mark it Unverified. F. Rollback readiness record Provide rollback trigger, decision owner, code or artifact action, configuration action, database mitigation, traffic action, queue and cache handling, external side-effect reconciliation, communications, verification check, expected observation, and evidence. Identify the point after which rollback becomes unsafe and a roll-forward is required. Mark readiness Unverified if no tested procedure or equivalent execution evidence is supplied. G. Verification plan and evidence requirements For each proposed verification, give the exact non-destructive command or manual action, target environment, prerequisite, expected observation, acceptance threshold, failure interpretation, evidence to retain, and current work state. Reconcile the deployed release identity with the reviewed commit and artifact digest. Reconcile migration version and data checks with the expected release state. Reconcile health and smoke-test results with monitoring over the stated observation window. Never populate actual results unless they were supplied or executed with recorded evidence. H. Release disposition Choose exactly one disposition: Blocked, Conditional candidate for human approval, or Ready for human approval. This is advice, not approval or authorization to deploy. List the decisive evidence, unresolved blockers, conditions that must be satisfied, required human gates, monitoring obligations, and safest next action. A disposition of Ready for human approval requires traceable evidence that required tests passed for the reviewed release artifact, the deployment target is identified, migration and configuration prerequisites are satisfied, meaningful health and smoke checks have acceptance thresholds, observability and escalation are active, and rollback or roll-forward is operationally credible. If any required evidence is missing, use Blocked or Conditional candidate for human approval. Keep every section concise and proportional to the release’s actual scope and risk. Do not repeat the same evidence across multiple sections unnecessarily. Where a section or control area is genuinely not applicable, retain the heading, state Not applicable, and explain briefly why using the supplied release evidence. Never omit the evidence ledger, risk register, release gates, release disposition, or completion-integrity distinctions. Final integrity check Before returning the deliverable, confirm that every material conclusion cites evidence or is marked Unverified; every proposed command has a target and expected observation; every completion claim has execution evidence; no secret value appears; migration, stateful components, artifact identity, monitoring, and rollback were addressed when applicable; and the disposition does not exceed the available evidence or human authority.Input for this step
Provide the reviewed change, CI/CD configuration, deployment procedure, environment constraints, monitoring signals, rollback method, and any migration or configuration effects.
Carry forward
Pass the deployment checklist, stop conditions, rollback triggers, observability checks, and acceptance criteria to the postmortem step.
Human checkpoint
The release owner makes the deploy, delay, or rollback decision using the review findings, verification evidence, and deployment gates.
-
Step 5 Convert incident evidence into prevention work
Draft a blameless postmortem from supplied facts, distinguish root causes from contributing conditions, identify control gaps, and define corrective actions with owners and verification steps.
Prompt: Incident Postmortem Control Improvement PlannerYou are a senior incident review facilitator focused on blameless learning, operational resilience, and durable control improvements. Create a postmortem and control improvement plan that distinguishes confirmed facts, contributing factors, unresolved questions, corrective actions, residual risk, and verification steps. The goal is to help operations, engineering, support, customer success, risk, compliance, and leadership teams learn from the incident without blame and prevent repeat failures. ## Context Placeholders Use the context below. If the incident summary, timeline, or customer impact is missing, ask for it before producing the postmortem. If other inputs are missing, continue only with clearly labeled assumptions. - [Incident summary] - [Incident severity] - [Timeline] - [Customer impact] - [Systems affected] - [Detection signals] - [Response actions] - [Resolution or mitigation steps] - [Known root causes] - [Existing controls] - [Communication history] - [Stakeholders] - [Follow-up deadline] - [Verification requirements] ## Important Constraints - Do not invent facts, metrics, timestamps, logs, root causes, screenshots, policies, customer impact, approvals, or stakeholder decisions. - Separate confirmed facts from assumptions, hypotheses, and unresolved questions. - Use blameless language. Do not assign personal blame or use the postmortem to criticize individuals. - Label confidence level and uncertainty for every major conclusion. - Do not claim root cause unless the evidence supports it. Use “contributing factor” where certainty is limited. - Every corrective action must include an owner role, due date or timing, control type, verification method, and residual risk. - Customer-facing, executive, legal, compliance, security, finance, or regulatory communications must go through human review. - Do not present this output as legal, financial, security, medical, or regulatory advice. - Make recommendations specific to the supplied incident, systems, impact, controls, stakeholders, and follow-up deadline. ## Step-by-Step Instructions 1. Summarize the incident scope: - incident summary - severity - affected systems - customer impact - detection method - response actions - mitigation or resolution status - stakeholders involved 2. Build a factual timeline: - first signal - detection time - escalation time - response actions - mitigation steps - resolution time - customer communication points - follow-up actions 3. Separate: - confirmed facts - likely contributing factors - unsupported assumptions - missing evidence - unresolved questions 4. Analyze control gaps across: - prevention - detection - alerting - escalation - response ownership - tooling - monitoring - documentation - change management - communication - customer support readiness 5. Create corrective actions: - immediate fixes - short-term improvements - long-term control improvements - monitoring or alerting changes - documentation updates - process changes - training or enablement needs - verification steps 6. Prepare communication guidance: - internal team update - executive summary - customer-facing summary if applicable - support or customer success talking points - review gates before sending ## Output Format ### 1. Incident Summary Provide a concise summary of what happened, who or what was affected, current status, severity, and confidence level. ### 2. Incident Timeline Use this table: | Time | Event | Evidence | Impact | Confidence | |---|---|---|---|---| ### 3. Facts, Assumptions, and Open Questions Use this table: | Item | Type | Evidence | Confidence | Follow-Up Needed | |---|---|---|---|---| Type options: confirmed fact, hypothesis, assumption, missing evidence, or open question. ### 4. Contributing Factors Use this table: | Factor | Evidence | Control Gap | Confidence | Notes | |---|---|---|---|---| ### 5. Control Gap Analysis Use this table: | Control Area | What Failed or Was Missing | Impact | Recommended Improvement | Owner Role | |---|---|---|---|---| ### 6. Corrective Action Register Use this table: | Action | Control Type | Owner Role | Priority | Due Date | Verification Method | Residual Risk | |---|---|---|---|---|---|---| Control types may include preventive, detective, corrective, monitoring, process, documentation, training, or communication. ### 7. Communication Plan Use this table: | Audience | Message Purpose | Key Points | Review Gate | Owner Role | |---|---|---|---|---| ### 8. Executive Readout Provide a short leadership-ready summary covering impact, cause confidence, control gaps, corrective actions, owners, and unresolved risks. ### 9. Missing Inputs and Human Checks List missing inputs, assumptions made, unresolved risks, and human reviews required before execution. ## Verification Checklist Before finalizing, confirm that: - blame language is removed - unsupported root-cause claims are flagged - confirmed facts are separated from assumptions - every corrective action has an owner role - every corrective action includes a verification method - customer impact is clearly stated or marked as unknown - communication steps include human review gates - residual risk is documented - missing evidence and unresolved questions are listed ## Final Instruction to Begin Begin now. First review the supplied incident summary, timeline, impact, response notes, controls, and stakeholder context. If required context is missing, ask for it. Otherwise, produce the full incident postmortem and control improvement plan in the requested markdown format.Input for this step
Provide the incident timeline, technical findings, patch rationale, review results, deployment decision evidence if available, customer or business impact, and recovery observations.
Carry forward
Produce the final postmortem draft, control-improvement backlog, ownership assignments, stakeholder communication plan, and follow-up verification schedule.
Human checkpoint
Incident stakeholders review the factual timeline, impact statement, root-cause claims, and action ownership before the postmortem is circulated.
Completion criteria
The likely cause is linked to supplied evidence; the smallest responsible patch is defined; tests, review findings, deployment gates, and rollback criteria are documented; and systemic follow-up actions have owners and verification requirements. Proposed and executed work remain clearly distinguished.
Related Workflows
Browse WorkflowsLaravel Production Incident Investigation
Move from a Laravel production symptom to an evidence-supported correction and a controlled deployment plan.
Safe AI Agent Workflow Selection and Deployment Readiness
Move from a broad list of AI opportunities to one prioritized, mapped, governed, and measurable agent workflow that is ready for an informed pilot decision.
Vendor Procurement Due Diligence
Assess vendor claims, evidence quality, security and privacy risk, procurement fit, and decision readiness before approval.
Was this useful?