Evidence-Based AI Workflow ROI Measurement Plan

Create an evidence-based measurement plan for the AI-assisted workflow described below. The deliverable must enable a decision owner to distinguish gross productivity claims from measured, quality-adjusted net value.

## Inputs

### Minimum inputs for a measured ROI conclusion
- Workflow description: [Workflow description]
- Current baseline: [Current baseline]
- Time or cost inputs: [Time or cost inputs]
- Quality metrics: [Quality metrics]
- Measurement period: [Measurement period]
- Data sources: [Data sources]
- Review or approval effort: [Review or approval effort]
- Rework rate or error rate: [Rework rate or error rate]
- Workflow owner: [Workflow owner]

### Decision and operating context
- Expected benefit: [Expected benefit]
- Users involved: [Users involved]
- Risk controls: [Risk controls]
- Decision threshold: [Decision threshold]
- Training and maintenance effort: [Training and maintenance effort]
- Adoption signals: [Adoption signals]

Treat a measurable baseline or a credible method for collecting one, a defined workflow unit, comparable quality evidence, and attributable cost or labor data as prerequisites for a measured ROI conclusion. Other missing inputs may permit a planning-only deliverable.

## General AI operating boundaries

Use General AI to organize supplied evidence, expose inconsistencies, show calculations, design the measurement approach, and draft decision rules. Analyze only information included in the conversation or attached source materials that are actually available.

Do not claim access to workflow systems, analytics platforms, financial records, employee activity, customer data, model logs, or control evidence unless their contents are supplied. Do not claim to have run a pilot, interviewed users, validated records, tested controls, approved a business case, changed a workflow, or deployed or stopped a system. These are human or system actions outside this analysis.

The output is decision support, not authorization. Scaling, pausing, stopping, changing controls, committing funds, using employee-level monitoring, or changing a high-impact workflow requires approval from the named decision owner and relevant privacy, security, legal, compliance, finance, HR, or domain reviewers.

Do not expose unnecessary personal, customer, confidential, regulated, credential, or security-sensitive data. Recommend aggregated or de-identified evidence wherever task-level data is sufficient.

## Evidence and missing-input rules

Classify every material input or conclusion as one of the following:
- Supplied fact: directly stated in a supplied source.
- Observed result: recorded outcome from supplied baseline or pilot evidence.
- Derived value: arithmetic calculated from cited supplied values.
- Assumption: a temporary value or interpretation requiring validation.
- Hypothesis: an explanation or expected effect not yet tested.
- Unknown: information unavailable from the supplied materials.
- Conflict: supplied sources disagree.
- Proposed: a metric, control, threshold, or action that has not been implemented.

Cite the source name, record, report, or user statement for each material figure. Preserve unknowns rather than inventing values. Never convert an assumption into an observed result.

If a prerequisite is absent or conflicting, first ask concise clarification questions. Then make bounded progress by producing a planning-only measurement design with formulas and collection steps, leaving numerical results uncalculated. If the user supplied evidence but it is insufficiently comparable, label the conclusion unverified and explain the mismatch. Never present a proposed threshold as approved.

## Analysis workflow

### 1. Define the decision and unit of analysis
Establish:
- The workflow boundary, trigger, endpoint, output, and excluded activities.
- The unit being measured, such as one accepted report, resolved case, reviewed document, or completed transaction.
- The decision to be made and the accountable decision owner.
- The baseline and AI-assisted variants being compared.
- The measurement window, user population, workflow volume, and evidence coverage.
- Whether the comparison is historical, before-and-after, matched cohort, randomized pilot, phased rollout, or another design.

Flag scope drift, denominator changes, volume differences, seasonal effects, staffing changes, demand mix changes, and simultaneous process changes that could invalidate the comparison.

### 2. Build an evidence register
Create an evidence register with these columns:
- Evidence ID
- Claim or metric supported
- Classification
- Source and date
- Population and measurement window
- Collection method
- Owner
- Reliability limitations
- Conflict or missing-data note
- Whether independent validation is required

Identify unsupported benefit claims, stale baselines, self-reported estimates, incomplete time tracking, survivorship bias, selection bias, excluded failures, and missing control evidence.

### 3. Reconstruct the baseline
Map each baseline process step and record:
- Role or system performing it
- Touch time and elapsed time
- Loaded labor cost or other attributable cost
- Workflow volume
- Review and approval effort
- Error, rejection, escalation, and rework rates
- Accepted-output rate
- Quality level and measurement method
- Customer or stakeholder effect
- Bottlenecks and exceptions
- Source, evidence classification, and confidence

Separate measured values from estimates. Show whether costs are fixed, variable, one-time, recurring, avoidable, or merely reallocated. Do not treat released capacity as cash savings unless the supplied evidence shows that spending was actually avoided or capacity produced validated incremental value.

### 4. Model the AI-assisted workflow
Map every changed, added, and removed step. Include:
- AI generation or assistance time
- Human prompt, preparation, and handling time
- Mandatory review and approval time
- Rework, regeneration, correction, and escalation time
- Training, onboarding, monitoring, maintenance, and governance effort
- Tool, integration, infrastructure, and vendor costs
- Failure handling and manual fallback
- Quality effects and risk exposure
- Adoption and bypass behavior
- Evidence source and confidence

Separate one-time implementation costs from recurring operating costs. State the amortization period if one-time costs are allocated across workflow units, and label that period as supplied or assumed.

### 5. Define and calculate the metrics
Use consistent units, periods, populations, and denominators. Show formulas before results and provide a calculation trace using evidence IDs.

At minimum, evaluate:
- Gross hours avoided = comparable baseline labor hours minus unchanged AI-assisted production hours.
- Net hours saved = gross hours avoided minus added preparation, review, rework, escalation, monitoring, training allocation, and recurring maintenance hours.
- Validated benefit = attributable labor value of net hours saved plus evidenced incremental value or avoided loss, without double counting.
- Incremental cost = AI fees plus integration, infrastructure, implementation allocation, governance, monitoring, and other attributable costs not already represented in labor adjustments.
- Net benefit = validated benefit minus incremental cost.
- ROI = net benefit divided by incremental cost, when incremental cost is positive and both numerator and denominator are adequately evidenced.
- Cost per accepted output = total attributable workflow cost divided by outputs meeting the quality acceptance standard.
- Quality-adjusted throughput = accepted outputs divided by total labor hours.
- Adoption rate = eligible workflow units completed through the approved AI-assisted path divided by all eligible workflow units.

Also evaluate quality score, defect rate, rework rate, escalation rate, customer or stakeholder impact, user satisfaction, risk incidents, control failures, and maintenance burden.

Do not calculate ROI from invented numbers. If a denominator is zero, ambiguous, or incomparable, mark the metric not calculable. Distinguish cash savings, productive capacity released, cost avoidance, revenue contribution, and qualitative benefit. Present sensitivity ranges only when their bounds and rationale are explicit.

### 6. Design a credible measurement method
Specify:
- Baseline and comparison design
- Inclusion and exclusion rules
- Sample size or workflow volume target and rationale
- Segmentation by task complexity, user group, exception type, or risk tier
- Data fields and collection methods
- Owners and collection cadence
- Quality scoring rubric and blind or independent review where appropriate
- Treatment of failed, abandoned, escalated, and manually completed cases
- Method for controlling learning effects, seasonality, novelty effects, and selection bias
- Planned analysis and reporting cadence
- Data retention, access, privacy, and minimization controls
- Stop conditions and fallback procedure

If no reliable baseline exists, propose a time-boxed baseline collection period before the AI comparison. If randomization is impractical, propose the strongest feasible comparison and explain the remaining attribution limits.

### 7. Establish quality and risk guardrails
Create a control table containing:
- Control ID
- Workflow risk or failure mode
- Preventive or detective control
- Metric and evidence source
- Frequency
- Control owner
- Proposed or approved pass threshold
- Human review requirement
- Escalation and stop condition
- Manual fallback or recovery action
- Actual observation, if supplied
- Status: pass, fail, unverified, or not applicable

Require qualified human review for legal, financial, medical, employment, security, compliance, regulated, public-facing, customer-impacting, or other high-impact outputs. Recommend pausing measurement or use when severe harm, unauthorized data exposure, material control failure, or unreliable output cannot be contained by the documented fallback.

### 8. Interpret adoption without mistaking it for value
For each adoption signal, state:
- Signal and denominator
- What it may indicate
- What it does not prove
- Possible gaming or misinterpretation
- Segments with low or high use
- Validation method
- Relationship to quality-adjusted value

Distinguish voluntary repeat use from mandated usage, experimentation, duplicate work, shadow processes, and use that creates downstream review burden.

### 9. Construct decision rules
Create rules for keep as-is, improve and retest, scale, pause, stop, replace, and require more human review. For every rule include:
- Required evidence
- Metric and threshold
- Minimum measurement window or volume
- Quality and risk guardrails that must also pass
- Confidence or uncertainty condition
- Decision owner and required reviewers
- Action if evidence is mixed

Use supplied approved thresholds where available. Otherwise provide clearly labeled proposed thresholds for human approval. Never recommend scale solely because time, usage, or satisfaction improved; quality and risk guardrails must pass, net value must be positive under the agreed definition, and material attribution limitations must be acceptable.

### 10. Verify and reconcile
Perform a verification matrix with these columns:
- Check
- Expected condition
- Actual observation from supplied evidence
- Evidence IDs
- Recalculation or reconciliation performed
- Status: pass, fail, unverified, or not applicable
- Unresolved issue and owner

Include these concrete checks:
1. Baseline and AI results use the same workflow boundary, unit, denominator, population, and comparable period.
2. Reported volumes reconcile to included, excluded, failed, escalated, and accepted outputs.
3. Gross time savings reconcile to net time savings after all added labor burdens.
4. Loaded labor rates, tool costs, and cost periods are traceable and use compatible units.
5. One-time and recurring costs are separated and not double counted.
6. Released capacity is not mislabeled as cash savings.
7. Quality scores use the same rubric and acceptance standard across variants.
8. Adoption uses eligible workflow units as its denominator and is not treated as proof of value.
9. Risk incidents and control failures are included rather than excluded as outliers.
10. ROI arithmetic can be reproduced from cited evidence IDs.
11. Decision thresholds are identified as approved or proposed and all required guardrails are evaluated.
12. Material assumptions, conflicts, exclusions, and attribution limitations remain visible.

A check passes only when the expected condition is supported by supplied evidence and any required reconciliation succeeds. If actual observations are unavailable, use unverified rather than pass. Do not state that the workflow was measured, validated, tested, approved, scaled, paused, stopped, or improved unless the supplied evidence demonstrates that action and outcome.

## Required deliverable

Return the analysis in this order:

### A. Decision brief
State the workflow, measurement maturity, decision being considered, decision owner, strongest evidence, largest uncertainty, and recommended status. Use one maturity label: planning only, partially measured, measured but unverified, or evidence-verified for this analysis. Use evidence-verified only when the relevant verification checks pass from supplied records.

### B. Input sufficiency and clarification
List prerequisites received, missing prerequisites, useful optional inputs missing, conflicts, clarification questions, and what analysis remains safe despite each gap.

### C. Workflow comparison
Provide side-by-side baseline and AI-assisted process tables, including time, cost, quality, review, rework, exceptions, controls, and evidence IDs.

### D. Evidence register
Provide the complete evidence register and identify unsupported claims.

### E. Metric dictionary and calculation ledger
For each metric, provide its definition, formula, numerator, denominator, period, source evidence IDs, calculation, result, unit, confidence, and limitation. Mark unavailable results not calculable.

### F. Measurement design
Provide the runnable collection and comparison plan, including owners, cadence, sampling, segmentation, bias controls, privacy protections, stop conditions, and fallback.

### G. Quality and risk control plan
Provide the control table, human review gates, escalation paths, recovery actions, and unresolved control gaps.

### H. Adoption interpretation
Provide adoption signals with denominators, limits, validation methods, and links to quality-adjusted value.

### I. Decision-rule matrix
Provide the keep, improve, scale, pause, stop, replace, and increased-review rules. Distinguish approved thresholds from proposed thresholds.

### J. Verification and reconciliation matrix
Report expected versus actual observations, evidence, reconciliation, status, and unresolved ownership for every required check.

### K. Recommendation and authorization handoff
Recommend keep, improve, scale, pause, stop, retest, or no decision yet. Include evidence used, evidence missing, threshold outcome, quality and risk outcome, confidence, alternatives considered, trade-offs, next measurement action, required human approvals, and conditions that would change the recommendation.

A recommendation is advisory and must not be described as approved or executed. If evidence does not support a decision, select no decision yet and identify the smallest credible next measurement step.

### L. Reporting template
Provide a reusable reporting table containing baseline result, AI-assisted result, variance, net time and cost impact, accepted-output quality, adoption denominator and rate, control result, confidence, decision status, unresolved issue, owner, and next review date.

End with a short completion-status statement that distinguishes analysis completed from evidence collection, validation, approval, and operational actions that remain proposed, unavailable, blocked, or unverified.