Qualitative Codebook Reliability and Reflexivity Protocol

Public field comparison

Qualitative Codebook Reliability and Reflexivity Protocol

Qualitative Codebook Reliability and Reflexivity Protocol

Build and stress-test a qualitative codebook using traceable excerpts, rival interpretations, coder disagreement, positionality and documented adjudication.

Build and stress-test a qualitative codebook using traceable excerpts, rival interpretations, coder disagreement, positionality and documented adjudication.

Use this when a qualitative research team needs a defensible codebook and disagreement protocol without treating AI-generated coding as researcher consensus.

Use this when a qualitative research team needs a defensible codebook and disagreement protocol without treating AI-generated coding as researcher consensus.

Qualitative Codebook Development
Coder Calibration
Qualitative Coding Reliability Review

Qualitative Codebook Development
Coder Calibration
Qualitative Coding Reliability Review

research_question_and_method
sample_material_and_draft_codebook
coding_team_and_reflexivity_context
ethics_privacy_and_project_constraints

research_question_and_method
sample_material_and_draft_codebook
coding_team_and_reflexivity_context
ethics_privacy_and_project_constraints

Use Claude for long qualitative materials, or another AI assistant capable of handling the full context. Paste only de-identified excerpts authorized for the selected tool, together with the method and any draft codebook or evolving coding notes. Run the prompt to prepare a provisional pack, then have the research team conduct the method-compatible human process: independent calibration and adjudication where coding consistency is intended, or reflexive analytic dialogue without forced consensus where it is not.

Use Claude for long qualitative materials, or another AI assistant capable of handling the full context. Paste only de-identified excerpts authorized for the selected tool, together with the method and any draft codebook or evolving coding notes. Run the prompt to prepare a provisional pack, then have the research team conduct the method-compatible human process: independent calibration and adjudication where coding consistency is intended, or reflexive analytic dialogue without forced consensus where it is not.

An interview study has a broad code called “trust” that different coders apply to competence, institutional legitimacy and interpersonal safety. The protocol separates the meanings, preserves a rival interpretation and defines a second calibration round.

An interview study has a broad code called “trust” that different coders apply to competence, institutional legitimacy and interpersonal safety. The protocol separates the meanings, preserves a rival interpretation and defines a second calibration round.

Expert

Expert

Claude

Claude

qualitative research

qualitative research

qualitative-research
research
research-methods
research-integrity
evidence-traceability
privacy-review

qualitative-research
research
research-methods
research-integrity
evidence-traceability
privacy-review

Qualitative Codebook Reliability and Reflexivity Protocol

Qualitative Codebook Reliability and Reflexivity Protocol

Build and stress-test a qualitative codebook with traceable excerpts, rival interpretations, coder disagreement, reflexivity and adjudication records.

Build and stress-test a qualitative codebook with traceable excerpts, rival interpretations, coder disagreement, reflexivity and adjudication records.

Prompt-body line comparison

Removed Added Unchanged context

−Develop or audit a qualitative codebook for a coding-reliability or structured codebook approach while preserving source context, interpretive plurality and researcher accountability. This protocol is not a substitute for reflexive thematic analysis methods that reject agreement as a quality objective.
+Develop or audit a method-compatible qualitative coding and reflexivity process while preserving source context, interpretive plurality and researcher accountability. Use a structured codebook and consistency process only when the supplied method calls for them; for reflexive thematic analysis, use evolving coding notes and analytic dialogue without treating agreement as a quality objective.
 
 ## Inputs
 
 Research question and qualitative method:
 {{research_question_and_method}}
 
 Sanitized sample material and any draft codebook or evolving coding notes:
 {{sample_material_and_draft_codebook}}
 
 Coding team and reflexivity context:
 {{coding_team_and_reflexivity_context}}
 
 Ethics, privacy and project constraints:
 {{ethics_privacy_and_project_constraints}}
 
 ## Working rules
 
 - Treat excerpts as supplied evidence; code meanings and themes are interpretations. Label assumptions, rival interpretations, evidence conflicts, missing context and unresolved uncertainty.
 - Preserve enough excerpt and document context to explain a code without exposing unnecessary participant identity or sensitive detail.
 - Do not invent quotations, participant characteristics, field context, code frequencies, coder decisions or agreement statistics.
−- Use this protocol only where a codebook and a coding-reliability, negotiated-agreement or structured comparison process fits the supplied method. If the project uses reflexive thematic analysis without coder-agreement or adjudication goals, stop and return a method-fit warning rather than imposing this protocol.
+- Select the method-compatible branch before proceeding. For coding-reliability, negotiated-agreement or structured-codebook approaches, use the specified codebook and comparison process. For reflexive thematic analysis, use evolving codes, analytic memos and documented dialogue without imposing independent agreement, adjudication, consensus or a fixed codebook.
 - The AI reviews supplied human coding records and excerpts. It does not act as an independent coder, and its suggestions are provisional analytic prompts. Never report them as coder agreement, researcher consensus or participant meaning.
 - Keep codebook changes versioned. The research team controls code adoption, theme construction and interpretive claims.
 
 ## Protocol
 
 1. Establish the methodological contract: epistemic stance, analytic unit, coding approach, role of prior theory, desired consistency and appropriate treatment of disagreement.
 2. Build an evidence inventory identifying excerpt IDs, source type, context available, consent or use restrictions, redactions and sampling limitations.
 3. Where the method uses a structured codebook, define every proposed code by name, concise meaning, inclusion criteria, exclusion criteria, close alternatives, positive and negative examples, contextual cautions and parent/child relationships. Where the method is reflexive and treats coding as evolving, maintain provisional coding notes and analytic memos instead; do not impose fixed inclusion/exclusion rules or a stable code hierarchy unless the team has explicitly justified them.
 
 4. Choose a human comparison process that follows the methodological contract. When coding consistency is a stated quality aim, have human coders apply the draft codebook independently to a deliberately varied calibration sample. For a reflexive approach, plan a documented analytic dialogue using varied material without treating independent agreement as a quality target. If actual human coding or dialogue records are unavailable, specify the sample and recording sheet but mark the exercise `Not run`.
 
 5. Analyze only records produced by the method-compatible process. For a coding-consistency approach, record exact agreement, partial overlap, divergent interpretation, missed context and codebook ambiguity; calculate an agreement statistic only when methodologically justified and the required data are supplied. For a reflexive approach, document convergent and divergent readings and how dialogue develops the analysis; do not score agreement or treat convergence as accuracy.
 6. Generate rival interpretations for consequential excerpts. For each, identify supporting evidence, disconfirming evidence, positionality considerations and a discriminating question.
 7. Conduct reflexivity review: researcher roles, relationships to the setting, theoretical commitments, power, language and decisions that may shape attention or interpretation. Do not infer identity or bias that was not supplied.
 8. Make code-adoption or interpretive decisions only through a documented human researcher process. For a structured codebook, record retain, revise, split, merge, retire or leave plural. For a reflexive approach, record how analytic dialogue reshaped or retained interpretations without forcing consensus. Preserve the rationale and any dissent.
 
 9. Design the next method-compatible calibration or reflexive review round and its stopping condition. Do not claim saturation, reliability or interpretive adequacy solely because the codebook is detailed, an agreement score is high or researchers reached consensus.
 
 ## Output contract: Coding and Reflexivity Pack
 
 Return:
 
 1. **Method fit statement**: intended analytic approach, appropriate consistency goal and prohibited overclaims.
+
 2. **Source and privacy ledger**: excerpt ID, context, consent/use boundary, redaction and analytical limitation.
+
 3. **Versioned coding record**: for a structured approach, code ID, label, definition, include/exclude rules, examples, near-neighbour distinctions and version note; for a reflexive approach, evolving code, source context, analytic memo, revision history and relationship to candidate themes.
 
 4. **Coding comparison record**: material, human coder labels or readings when supplied, method-compatible comparison, evidence, contextual insight and decision status. Include an agreement statistic only when methodologically justified and actually calculated from supplied data.
 
−5. **Human decision log**: decision or interpretive development, participants, evidence considered, dissent or retained plurality, resulting change and effective version.
+5. **Reflexivity register**: standpoint or process factor, supplied basis, possible analytic effect, counter-practice and unresolved question.
 
−6. **Next-round plan**: sample or material logic, human analytic steps, method-compatible comparison or dialogue, review roles and repeat/stop rules.
+6. **Human decision log**: decision or interpretive development, participants, evidence considered, dissent or retained plurality, resulting change and effective version.
 
−7. **Status note**: distinguish drafted, calibrated, revised, agreed, plural and untested elements.
+7. **Next-round plan**: sample or material logic, human analytic steps, method-compatible comparison or dialogue, review roles and repeat/stop rules.
 
+8. **Status note**: distinguish drafted, calibrated, revised, agreed, plural and untested elements.
+
 ## Completion conditions
 
 The pack is complete only when coding records preserve source context, examples trace to authorized excerpts, interpretive differences and dissent remain visible, the comparison or dialogue process fits the qualitative tradition, and adopted decisions identify the responsible researchers. Require clear distinctions between neighbouring codes only where the stated method uses a structured codebook. Do not require agreement metrics, independent coding, consensus or a fixed codebook for reflexive methods. If source context, consent boundaries or the required human analytic evidence are missing, mark the affected parts incomplete and request them.
 
 For each completion condition, record the expected evidence, the actual supplied evidence, `Met`, `Not met`, or `Blocked` status, and any unresolved researcher action.
 
 Do not certify thematic saturation, intercoder reliability, validity, consensus or ethical compliance unless the named procedure was actually completed and evidenced.