Agent Escalation Threshold Calibration

Calibrate escalation thresholds for an AI agent using evidence about severity, uncertainty, control failures, reviewer capacity, delay, and actual outcomes. Distinguish thresholds that pause work from those that route routine review or trigger urgent containment.

Provide:
- Agent tasks, user groups, decisions/actions, risk classes, protected outcomes, and unacceptable failures: [Agent decisions and risk classes]
- Rules, scores, confidence/uncertainty signals, triggers, routes, priorities, context package, and fallback behavior: [Current escalation logic and routes]
- Incidents, near misses, reviewer decisions, overrides, missed escalations, unnecessary escalations, user requests, and downstream outcomes: [Incident feedback and outcome evidence]
- Score distributions, calibration, labels, disagreement, slices, drift, missingness, and historical threshold changes: [Score uncertainty and threshold evidence]
- Arrival volume, service targets, reviewer skill, coverage hours, queue age, abandonment, and surge capacity: [Queue capacity and service constraints]
- Non-delegable approvals, service owner, product owner, review owner, security/privacy reviewer, and emergency authority: [Authority boundaries and accountable owners]

Do not invent score distributions, error rates, or queue behavior. Distinguish observed outcome evidence from inference. Do not optimize for fewer escalations without pricing missed harm, or for maximum sensitivity without considering delay and reviewer overload. Separate uncertain model scores from independently observed risk triggers. Preserve user-requested escalation and mandatory policy triggers even if statistical tuning suggests otherwise.

Calibration:

1. Define escalation classes.
   Specify Continue, Clarify, Abstain, Routine review, Priority review, Immediate pause/containment, and Emergency response as applicable. State purpose, owner, service target, and allowed agent behavior while waiting.

2. Build the outcome ledger.
   Reconcile current trigger decisions with reviewer outcomes, incidents, corrections, and downstream consequences. Identify true positive, false positive, true negative, false negative, and not-assessable cases only where labels support them.

3. Audit signals.
   Review definition, calibration, stability, missing values, manipulability, protected slices, and independence. Identify hard rules that must override probabilistic thresholds.

4. Model threshold trade-offs.
   Compare candidate thresholds using supplied confusion, severity, workload, and delay evidence. Show how volume and queueing change, and identify where performance is not estimable.

5. Protect critical slices and events.
   Define zero- or low-tolerance triggers for irreversible action, sensitive data, safety, legal/compliance, identity/authorization, user distress, or explicit review requests where applicable. Assign qualified routes.

6. Test operational feasibility.
   Check reviewer capacity, skills, hours, context quality, queue priority, fallback, and outage behavior. A threshold is not deployable if the designated route cannot meet the required response.

7. Recommend thresholds and rollout.
   Specify thresholds, hard triggers, hysteresis or cooldown if needed, context package, staged rollout, shadow comparison, monitoring, drift and queue alerts, stop conditions, and recalibration cadence.

Use ChatGPT to analyze the supplied incidents, thresholds, and reviewer-capacity evidence; do not claim simulation, deployment, or threshold validation occurred unless results are supplied. Keep proposed threshold changes distinct from approved configuration, and identify the accountable operations owner and release owner for authorization.

Required deliverable:

# Agent Escalation Threshold Calibration

## Escalation Classes and Authority
| Class | Trigger purpose | Agent behavior while pending | Route/owner | Service target | Mandatory rule |
|---|---|---|---|---|---|

## Outcome and Error Ledger
| Slice/trigger | Escalations | Confirmed needed | Unnecessary | Missed | Consequence | Evidence quality |
|---|---:|---:|---:|---:|---|---|

## Signal Fitness
| Signal | Meaning | Calibration/stability | Missing/manipulation risk | Fit for threshold? |
|---|---|---|---|---|

## Candidate Thresholds
| Slice/class | Threshold/rule | Expected miss/over-route effect | Queue impact | Risk | Evidence limit |
|---|---|---|---|---|---|

## Recommended Policy
| Slice/event | Continue threshold | Review threshold | Pause/containment trigger | Route | Owner |
|---|---|---|---|---|---|

## Rollout and Recalibration
- Shadow/canary plan:
- Queue and incident stop conditions:
- Protected hard rules:
- Drift/recalibration triggers:
- Evidence still needed:

Completion requires outcome-linked thresholds, feasible review routes, preserved mandatory authority boundaries, and monitoring that detects both missed harm and overload.