Review an Audit Sampling and Control-Test Design
Challenge a proposed audit sampling and control-test design for population coverage, selection logic, evidence requirements, exception handling and defensible inference before testing begins.
Category
Data Analysis
Prompts for auditing datasets, planning SQL analysis, defining dashboards, reviewing experiments, and checking causal assumptions.
Automate KPI Reporting from Approved Metric Definitions
Implement a repeatable KPI-reporting process from approved metric definitions, with traceable calculations, authoritative reconciliation, failure and rerun tests, disabled delivery, and recovery evidence.
Knowledge Entitlement Drift Review
Reconcile authoritative access policy with effective permissions across source, ingestion, index, cache, retrieval, citation, and response layers.
Forecast Assumption and Model Drift Challenge
Challenge forecast assumptions, structural stability, backtest evidence, scenario sensitivity, and decision thresholds before relying on projected outcomes.
Data Lineage Break Investigation
Reconstruct where a data product diverged from authoritative lineage, bound affected outputs and decisions, and define safe repair and reprocessing.
Analytical Conclusion Sensitivity Review
Test whether a consequential analytical conclusion survives plausible changes to data, cohort, definitions, assumptions, model choices, and missing-information treatment.
RAG Access-Control Leakage Investigation
Investigate whether a RAG system crossed identity, tenant, purpose, region, or document authorization boundaries during retrieval, embedding, caching, citation, or response exposure.
Retrieval Chunking and Metadata Experiment Design
Design a controlled retrieval experiment that compares chunking and metadata choices without turning into a full RAG redesign.
AI-Generated SQL Result Verification and Reconciliation
Validate AI-generated SQL and its reported results before they are used for a consequential decision.
AI Trace Review and Failure Taxonomy
Use ChatGPT to reconstruct evidence-supported AI and agent traces, govern failure classifications, analyze recurring patterns within sampling limits, identify observability gaps, and propose sanitized regression cases and measurable prevention work.
AI Quality, Latency, and Cost Trade-off Experiment
Design a controlled AI workflow experiment that compares task quality, tail latency, reliability, token and tool cost, uncertainty, and operational constraints.
dbt Model Test Coverage and Lineage Review
Review dbt test coverage, contracts, freshness, lineage, artifacts, CI selection, warehouse risk, and focused repairs without deploying.