Marking Moderation Anomaly Investigation

Investigate potential marking anomalies using de-identified scorer and criterion evidence while preserving legitimate academic judgment and formal grade authority.

## Moderation evidence

Assessment, rubric, anchors and authorized marking rules:
{{assessment_rubric_and_rules}}

De-identified criterion-level scores and marker rationales:
{{scoring_and_rationale_evidence}}

Moderation process, marker allocation and relevant context:
{{moderation_process_and_context}}

Decision thresholds, exceptions and institutional constraints:
{{decision_rules_and_constraints}}

## Evidence discipline

- Separate observed score pattern, documented rationale, statistical signal, inference, assumption, conflict, missing information and unresolved uncertainty.
- Do not infer marker bias, learner merit, misconduct or protected characteristics from a distribution alone.
- Do not invent grades, rationales, anchor decisions, sample sizes, agreement measures or institutional thresholds.
- Use pseudonymous IDs when records remain linkable and the minimum data needed. Apply small-cell suppression and re-identification controls; avoid exposing names, disability information, integrity cases or unrelated learner records.
- Make every numerical comparison reproducible through recorded formulas or code. Reconcile source and analysis row counts, use methods appropriate to the scoring scale and sample size, report denominators and uncertainty, and require an independent numerical check before consequential action.
- A statistical anomaly is a review trigger, not proof of incorrect marking. Preserve documented exceptions and subject-matter judgment.
- Do not alter grades or communicate decisions. The authorized moderation or assessment board owns consequential action.

## Investigation

1. Define the assessed construct, rubric version, scoring scale, markers, allocation process, moderation stage and decision that the analysis may inform.
2. Validate the evidence grain. Reconcile source and analysis row counts; check duplicates, missing scores, inconsistent scales, late rubric versions, marker reassignment, team-marking structure, small cells and non-comparable cohorts.
3. Compare overall and criterion-level patterns by marker using scale-appropriate methods for counts, location, spread, missingness, boundary use and score combinations. Report formulas or code, denominators and uncertainty, and flag results awaiting independent numerical checking.
4. Examine anchor or double-marked cases. Compare evidence cited, criterion interpretation, score difference and adjudication, not just the final totals.
5. Test plausible explanations: allocation mix, cohort or topic difference, rubric ambiguity, marker severity, criterion interaction, arithmetic or import defect, authorized adjustment and random variation.
6. Identify the earliest point where marking practice diverges from the agreed process or where evidence cannot distinguish explanations.
7. Select a proportionate response: no action, clarification, targeted sample review, recalibration, broader remarking proposal or data correction. Define triggers before recommending expansion.
8. Record affected decisions conservatively. Do not assume every script marked by a flagged scorer is wrong.

## Output contract: Moderation Anomaly Case File

Return:

1. **Scope and safeguards**: assessment, rubric version, data boundary, pseudonymization or aggregation, small-cell rule, authority and exclusions.
2. **Evidence quality check**: dataset field, grain, completeness, comparability issue and correction status.
3. **Pattern table**: marker/criterion, denominator, statistic or observable pattern, reference, uncertainty and signal status.
4. **Anchor decision comparison**: case ID, criterion evidence, scorer rationale, difference, adjudication and unresolved interpretation.
5. **Competing explanation matrix**: hypothesis, supporting evidence, contradictory evidence, confidence and discriminating check.
6. **Potential impact boundary**: affected assessment slice, why, what is not established and learner-risk control.
7. **Moderation action plan**: action, sample, owner, trigger, evidence to retain, stop condition and escalation route.
8. **Disposition**: `No material anomaly found`, `Targeted review required`, `Process correction required`, or `Insufficient evidence`, clearly separated from any grade decision.

## Completion conditions

Complete only when data comparability is checked, every anomaly includes its denominator and uncertainty, anchor evidence has been examined where available, plausible non-marker explanations are tested, and the recommended review scope is proportionate. If rationale, rubric version or allocation data are missing, mark the case incomplete.

Reject requests to automatically change grades, rank marker quality, infer discrimination from unsupported proxies or conceal a material process error. Refer formal decisions and learner communication to the authorized moderation process.