Marking Moderation Anomaly Investigation

Public field comparison

Marking Moderation Anomaly Investigation

Marking Moderation Anomaly Investigation

Investigate scorer patterns, criterion distributions, anchor decisions and justified exceptions for possible marking drift without automatically changing grades.

Investigate scorer patterns, criterion distributions, anchor decisions and justified exceptions for possible marking drift without automatically changing grades.

Use this when moderation evidence suggests inconsistent scoring and the academic team needs a criterion-level investigation and proportionate review plan.

Use this when moderation evidence suggests inconsistent scoring and the academic team needs a criterion-level investigation and proportionate review plan.

Post-Marking Moderation
Multi-Marker Drift Investigation
Assessment Board Evidence Preparation

Post-Marking Moderation
Multi-Marker Drift Investigation
Assessment Board Evidence Preparation

assessment_rubric_and_rules
scoring_and_rationale_evidence
moderation_process_and_context
decision_rules_and_constraints

assessment_rubric_and_rules
scoring_and_rationale_evidence
moderation_process_and_context
decision_rules_and_constraints

Use this prompt with any capable AI assistant, providing only de-identified aggregate and criterion-level data. Paste the rubric version, moderation rules and documented marker rationales; remove names and sensitive learner information. Run the investigation, then have the module leader, moderation lead and assessment board review any proposed sample expansion or grade action.

Use this prompt with any capable AI assistant approved by your institution. Provide only the minimum necessary aggregate or criterion-level data after applying institutional anonymization or pseudonymization, small-cell, retention and access rules. Remove names, identifying free text and unnecessary sensitive learner information. Paste the rubric version, moderation rules and redacted marker rationales. Run the investigation, then have the module leader, moderation lead and assessment board review any proposed sample expansion or grade action.

A module shows a ten-point mean difference between two markers. Criterion and allocation analysis reveals one marker assessed all specialist projects and used a clarified rubric version, while anchor cases still expose one genuine descriptor drift requiring targeted recalibration.

A module shows a ten-point mean difference between two markers. Criterion and allocation analysis reveals one marker assessed all specialist projects and used a clarified rubric version, while anchor cases still expose one genuine descriptor drift requiring targeted recalibration.

Expert

Expert

General AI

General AI

analysis

analysis

assessment
rubric
evaluation
evidence-based-analysis
bias-reduction
education

assessment
rubric
evaluation
evidence-based-analysis
assessment-quality
education

Marking Moderation Anomaly Investigation

Marking Moderation Anomaly Investigation

Investigate scorer drift through criterion distributions, anchors, rationales and competing explanations without automatically changing grades.

Investigate possible marking drift through criterion distributions, anchors, rationales and competing explanations without automatically changing grades.

Prompt-body line comparison

Removed Added Unchanged context

−Investigate potential marking anomalies using de-identified scorer and criterion evidence while preserving legitimate academic judgment and formal grade authority.
+Investigate potential marking anomalies using the minimum necessary scorer and criterion evidence after applying appropriate anonymization or controlled pseudonymization, while preserving legitimate academic judgment and formal grade authority.
 
 ## Moderation evidence
 
 Assessment, rubric, anchors and authorized marking rules:
 {{assessment_rubric_and_rules}}
 
−De-identified criterion-level scores and marker rationales:
+Minimized criterion-level scores and redacted marker rationales, with anonymization or pseudonymization controls:
 {{scoring_and_rationale_evidence}}
 
 Moderation process, marker allocation and relevant context:
 {{moderation_process_and_context}}
 
 Decision thresholds, exceptions and institutional constraints:
 {{decision_rules_and_constraints}}
 
 ## Evidence discipline
 
 - Separate observed score pattern, documented rationale, statistical signal, inference, assumption, conflict, missing information and unresolved uncertainty.
 - Do not infer marker bias, learner merit, misconduct or protected characteristics from a distribution alone.
 - Do not invent grades, rationales, anchor decisions, sample sizes, agreement measures or institutional thresholds.
−- Use pseudonymous IDs when records remain linkable and the minimum data needed. Apply small-cell suppression and re-identification controls; avoid exposing names, disability information, integrity cases or unrelated learner records.
+- Treat replacement IDs as pseudonyms unless an identifiability assessment shows that re-identification risk is sufficiently remote. Apply institutional data-minimization, small-cell, retention, access and approved-tool rules; remove identifying free text before analysis.
 - Make every numerical comparison reproducible through recorded formulas or code. Reconcile source and analysis row counts, use methods appropriate to the scoring scale and sample size, report denominators and uncertainty, and require an independent numerical check before consequential action.
+- If inferential statistics are used, state their assumptions and distinguish prespecified from exploratory comparisons. Account for small samples, clustered or repeated scores, non-random marker allocation and multiple comparisons.
 - A statistical anomaly is a review trigger, not proof of incorrect marking. Preserve documented exceptions and subject-matter judgment.
 - Do not alter grades or communicate decisions. The authorized moderation or assessment board owns consequential action.
 
 ## Investigation
 
 1. Define the assessed construct, rubric version, scoring scale, markers, allocation process, moderation stage and decision that the analysis may inform.
 2. Validate the evidence grain. Reconcile source and analysis row counts; check duplicates, missing scores, inconsistent scales, late rubric versions, marker reassignment, team-marking structure, small cells and non-comparable cohorts.
 3. Compare overall and criterion-level patterns by marker using scale-appropriate methods for counts, location, spread, missingness, boundary use and score combinations. Report formulas or code, denominators and uncertainty, and flag results awaiting independent numerical checking.
 4. Examine anchor or double-marked cases. Compare evidence cited, criterion interpretation, score difference and adjudication, not just the final totals.
 5. Test plausible explanations: allocation mix, cohort or topic difference, rubric ambiguity, marker severity, criterion interaction, arithmetic or import defect, authorized adjustment and random variation.
 6. Identify the earliest point where marking practice diverges from the agreed process or where evidence cannot distinguish explanations.
 7. Select a proportionate response: no action, clarification, targeted sample review, recalibration, broader remarking proposal or data correction. Define triggers before recommending expansion.
 8. Record affected decisions conservatively. Do not assume every script marked by a flagged scorer is wrong.
 
 ## Output contract: Moderation Anomaly Case File
 
 Return:
 
 1. **Scope and safeguards**: assessment, rubric version, data boundary, pseudonymization or aggregation, small-cell rule, authority and exclusions.
 2. **Evidence quality check**: dataset field, grain, completeness, comparability issue and correction status.
 3. **Pattern table**: marker/criterion, denominator, statistic or observable pattern, reference, uncertainty and signal status.
 4. **Anchor decision comparison**: case ID, criterion evidence, scorer rationale, difference, adjudication and unresolved interpretation.
 5. **Competing explanation matrix**: hypothesis, supporting evidence, contradictory evidence, confidence and discriminating check.
 6. **Potential impact boundary**: affected assessment slice, why, what is not established and learner-risk control.
 7. **Moderation action plan**: action, sample, owner, trigger, evidence to retain, stop condition and escalation route.
 8. **Disposition**: `No material anomaly found`, `Targeted review required`, `Process correction required`, or `Insufficient evidence`, clearly separated from any grade decision.
 
 ## Completion conditions
 
−Complete only when data comparability is checked, every anomaly includes its denominator and uncertainty, anchor evidence has been examined where available, plausible non-marker explanations are tested, and the recommended review scope is proportionate. If rationale, rubric version or allocation data are missing, mark the case incomplete.
+Complete only when data comparability is checked, every anomaly includes its denominator and uncertainty, any inferential analysis states its assumptions and exploratory or confirmatory status, anchor evidence has been examined where available, plausible non-marker explanations are tested, and the recommended review scope is proportionate. If rationale, rubric version or allocation data are missing, mark the case incomplete.
 
 Reject requests to automatically change grades, rank marker quality, infer discrimination from unsupported proxies or conceal a material process error. Refer formal decisions and learner communication to the authorized moderation process.