Reusable AI capability
Recover Failed Kubernetes Rollouts Safely
Apply a repeatable evidence and decision method to isolate a failed Kubernetes rollout, choose the smallest safe recovery path, and verify restored service before closing the incident.
This Skill is a reusable set of instructions based on the linked Amo.ng Prompt or Workflow. It tells you what to provide, how to apply the method and how to know when the task is complete. It does not run automatically.
# Recover Failed Kubernetes Rollouts Safely Skill ID: AMO-S-000017 Skill URL: https://amo.ng/skills/recover-failed-kubernetes-rollouts-safely Purpose: Give platform and service teams a reusable rollout-recovery capability that separates diagnosis, recovery authorization, execution evidence, and service verification across different Kubernetes workloads and release mechanisms. Required inputs: - Affected environment, namespace, workloads, services, and incident window - Release, manifest, image, configuration, secret, ingress, service, policy, or infrastructure changes - Available rollout status, events, logs, probes, endpoints, routing, metrics, alerts, and application evidence - User impact, SLO constraints, recovery window, rollback and forward-fix options - Platform, service, security, incident, and release owners plus authorization boundaries How to use: When to use: - A Kubernetes rollout is failed, stalled, or degraded and the team needs an evidence-based recovery decision. - The same recovery discipline must be applied consistently across Deployments, StatefulSets, Services, ingress, probes, configuration, or dependencies. When not to use: - Designing a new cluster architecture or generic Kubernetes hardening. - Executing production changes without the relevant platform and release authority. - Declaring recovery when workload and application evidence are unavailable. Reusable method: 1. Freeze the incident boundary, latest known-good release, impact, recent changes, and inspectable evidence. 2. Build a timeline and classify observations, inferences, assumptions, missing evidence, and actions already taken. 3. Localize failure across artifact, scheduling, configuration, secret, lifecycle, probe, service discovery, ingress, network, dependency, resource, and autoscaling layers. 4. Rank competing causes and define the next discriminating observation for unresolved hypotheses. 5. Compare rollback, configuration reversal, image correction, probe correction, scaling, traffic diversion, dependency recovery, and continue-investigation paths. 6. Select the smallest safe path with prerequisites, expected effect, blast radius, stop condition, rollback or compensation option, and authorized owner. 7. Verify workload availability, pod stability, probes, endpoints, routing, application transactions, metrics, logs, and customer-impact indicators over a defined monitoring window. Expected output: A recovery decision record containing evidence timeline, affected resources, layered hypothesis register, chosen and rejected recovery paths, authorization gates, execution checklist, service-verification matrix, monitoring window, unresolved evidence, and owner handoff. Boundaries: Do not invent cluster state, command output, changes, approvals, or recovery. Platform and service owners authorize configuration and workload actions; the security reviewer dispositions material security impact; the incident and release owners control production recovery and closure. AMO-P-000235 is the grounding source, not a prerequisite to invoke the Skill. Based on Prompt: Kubernetes Deployment Recovery Runbook Source ID: AMO-P-000235 https://amo.ng/prompts/kubernetes-deployment-recovery-runbook Completion criteria: Complete when: - Every material cause finding links to supplied evidence and unresolved causes have a discriminating check. - The chosen path has prerequisites, owner, expected effect, risk, stop condition, and fallback. - Production execution and observed results are distinguished from proposed commands. - Workload and application verification have observable results or remain explicitly pending. - The status is Ready for authorized recovery, Conditionally ready, Blocked, or Recovered with evidence. Use this Amo.ng Skill with your preferred AI tool. Supply the required inputs and follow the usage instructions. # Recover Failed Kubernetes Rollouts Safely Skill ID: AMO-S-000017 Skill URL: https://amo.ng/skills/recover-failed-kubernetes-rollouts-safely Purpose: Give platform and service teams a reusable rollout-recovery capability that separates diagnosis, recovery authorization, execution evidence, and service verification across different Kubernetes workloads and release mechanisms. Required inputs: - Affected environment, namespace, workloads, services, and incident window - Release, manifest, image, configuration, secret, ingress, service, policy, or infrastructure changes - Available rollout status, events, logs, probes, endpoints, routing, metrics, alerts, and application evidence - User impact, SLO constraints, recovery window, rollback and forward-fix options - Platform, service, security, incident, and release owners plus authorization boundaries How to use: When to use: - A Kubernetes rollout is failed, stalled, or degraded and the team needs an evidence-based recovery decision. - The same recovery discipline must be applied consistently across Deployments, StatefulSets, Services, ingress, probes, configuration, or dependencies. When not to use: - Designing a new cluster architecture or generic Kubernetes hardening. - Executing production changes without the relevant platform and release authority. - Declaring recovery when workload and application evidence are unavailable. Reusable method: 1. Freeze the incident boundary, latest known-good release, impact, recent changes, and inspectable evidence. 2. Build a timeline and classify observations, inferences, assumptions, missing evidence, and actions already taken. 3. Localize failure across artifact, scheduling, configuration, secret, lifecycle, probe, service discovery, ingress, network, dependency, resource, and autoscaling layers. 4. Rank competing causes and define the next discriminating observation for unresolved hypotheses. 5. Compare rollback, configuration reversal, image correction, probe correction, scaling, traffic diversion, dependency recovery, and continue-investigation paths. 6. Select the smallest safe path with prerequisites, expected effect, blast radius, stop condition, rollback or compensation option, and authorized owner. 7. Verify workload availability, pod stability, probes, endpoints, routing, application transactions, metrics, logs, and customer-impact indicators over a defined monitoring window. Expected output: A recovery decision record containing evidence timeline, affected resources, layered hypothesis register, chosen and rejected recovery paths, authorization gates, execution checklist, service-verification matrix, monitoring window, unresolved evidence, and owner handoff. Boundaries: Do not invent cluster state, command output, changes, approvals, or recovery. Platform and service owners authorize configuration and workload actions; the security reviewer dispositions material security impact; the incident and release owners control production recovery and closure. AMO-P-000235 is the grounding source, not a prerequisite to invoke the Skill. Based on Prompt: Kubernetes Deployment Recovery Runbook Source ID: AMO-P-000235 https://amo.ng/prompts/kubernetes-deployment-recovery-runbook Completion criteria: Complete when: - Every material cause finding links to supplied evidence and unresolved causes have a discriminating check. - The chosen path has prerequisites, owner, expected effect, risk, stop condition, and fallback. - Production execution and observed results are distinguished from proposed commands. - Workload and application verification have observable results or remain explicitly pending. - The status is Ready for authorized recovery, Conditionally ready, Blocked, or Recovered with evidence.Copy Skill copies the full Skill instructions. Use with AI copies those instructions with a short guide for your chosen AI tool. Both actions only copy text—they do not send anything to an AI provider or run the Skill.
Purpose
Give platform and service teams a reusable rollout-recovery capability that separates diagnosis, recovery authorization, execution evidence, and service verification across different Kubernetes workloads and release mechanisms.
Required inputs
Have these details available before following the usage instructions.
- Affected environment, namespace, workloads, services, and incident window
- Release, manifest, image, configuration, secret, ingress, service, policy, or infrastructure changes
- Available rollout status, events, logs, probes, endpoints, routing, metrics, alerts, and application evidence
- User impact, SLO constraints, recovery window, rollback and forward-fix options
- Platform, service, security, incident, and release owners plus authorization boundaries
How to use this Skill
When to use:
- A Kubernetes rollout is failed, stalled, or degraded and the team needs an evidence-based recovery decision.
- The same recovery discipline must be applied consistently across Deployments, StatefulSets, Services, ingress, probes, configuration, or dependencies.
When not to use:
- Designing a new cluster architecture or generic Kubernetes hardening.
- Executing production changes without the relevant platform and release authority.
- Declaring recovery when workload and application evidence are unavailable.
Reusable method:
1. Freeze the incident boundary, latest known-good release, impact, recent changes, and inspectable evidence.
2. Build a timeline and classify observations, inferences, assumptions, missing evidence, and actions already taken.
3. Localize failure across artifact, scheduling, configuration, secret, lifecycle, probe, service discovery, ingress, network, dependency, resource, and autoscaling layers.
4. Rank competing causes and define the next discriminating observation for unresolved hypotheses.
5. Compare rollback, configuration reversal, image correction, probe correction, scaling, traffic diversion, dependency recovery, and continue-investigation paths.
6. Select the smallest safe path with prerequisites, expected effect, blast radius, stop condition, rollback or compensation option, and authorized owner.
7. Verify workload availability, pod stability, probes, endpoints, routing, application transactions, metrics, logs, and customer-impact indicators over a defined monitoring window.
Expected output:
A recovery decision record containing evidence timeline, affected resources, layered hypothesis register, chosen and rejected recovery paths, authorization gates, execution checklist, service-verification matrix, monitoring window, unresolved evidence, and owner handoff.
Boundaries:
Do not invent cluster state, command output, changes, approvals, or recovery. Platform and service owners authorize configuration and workload actions; the security reviewer dispositions material security impact; the incident and release owners control production recovery and closure. AMO-P-000235 is the grounding source, not a prerequisite to invoke the Skill.
Based on an Amo.ng Prompt
Kubernetes Deployment Recovery Runbook
Open the linked Prompt to read the instructions this Skill is based on.
Completion criteria
Complete when:
- Every material cause finding links to supplied evidence and unresolved causes have a discriminating check.
- The chosen path has prerequisites, owner, expected effect, risk, stop condition, and fallback.
- Production execution and observed results are distinguished from proposed commands.
- Workload and application verification have observable results or remain explicitly pending.
- The status is Ready for authorized recovery, Conditionally ready, Blocked, or Recovered with evidence.
Was this useful?
Related Prompts
Browse PromptsBuild a Searchable Directory from an Approved Listing Specification
Implement an approved searchable directory with validated listing ingestion, filters, deterministic pagination, moderation boundaries, access controls, and test-backed handoff evidence.
Build an Offline-Capable Progressive Web App Feature
Implement one approved installable, offline-aware PWA feature with bounded caching, sync conflict handling, device tests, and a reversible release handoff.
Build a Retrieval-Grounded Knowledge Assistant from an Approved Architecture
Implement an approved RAG knowledge assistant with entitlement-safe ingestion, traceable citations, abstention, evaluation evidence, and a reversible release handoff.
Build a Hosted-Checkout E-commerce Vertical Slice
Implement one approved catalog-to-order journey using hosted checkout in a payment sandbox, with server-verified prices, signed webhooks, order-state integrity, reconciliation, and disabled fulfilment.
Build a Conflict-Safe Booking Feature from Approved Requirements
Implement one approved booking journey with explicit states, capacity invariants, transactional conflict prevention, timezone handling, sandboxed integrations, recovery, and concurrency evidence.
Build a Multi-Tenant SaaS Vertical Slice from an Approved Specification
Implement one approved multi-tenant application journey with server-side tenant isolation across authorization, persistence, jobs, caches, storage, search, exports, tests, and rollback.