# AI Vendor and Operational Control Review Workbook

Track A and Track B · Neutral documentation bundle

## Purpose

Use linked sheets to distinguish supplier claims, intended controls, effective behaviour and outstanding evidence across the operating lifecycle.

## Intended user

AI audit practitioner with procurement, access, operations, security and change owners supplying records; specialist decisions stay with authorised reviewers.

## When to use

When a scoped question concerns vendor controls, permissions, oversight, monitoring, an incident or a system change. Complete only relevant modules.

## Included templates

- **Vendor due-diligence questionnaire**.
- **Agent permission and tool-access review**.
- **Human override/control review**.
- **AI monitoring review**.
- **AI incident review**.
- **AI change-management review**.

## Shared evidence/provenance and human-review fields

Complete this cover once per working-paper record. Repeat only where the scope or reviewer changes. Local review rows should reference the shared decision ID when the same decision covers them; record only view-specific judgement or a different decision there, not duplicate names/dates. A role label alone may not identify the actual reviewer: use the responsible person or institution-controlled reviewer reference in the authorised record, without unnecessary personal data in provider inputs. Use source IDs in individual rows rather than copying the full evidence inventory. Blank is not a pass; use “not supplied”, “not performed”, “unclear”, or “not applicable — reason” accurately.

| Field | Definition / professional purpose | Entry |
|---|---|---|
| Record ID and objective | Local working-paper reference and the decision/question this record supports; not a newly assigned AMO ID | — |
| System/process and period | Exact unit, system baseline and period examined; bounds the conclusion | — |
| Source register | Evidence ID; permitted locator; version/date; source owner; date obtained; scope/access limitation. A cited document not inspected stays “not inspected” | — |
| Preparer and work state | Responsible preparer role; date; distinguish proposed, performed and supplied third-party work | — |
| Reviewer decision | Reviewer identity/reference and authority; date; exact record/version reviewed; decision ID and rationale; restrictions. Leave unrecorded until a person decides | — |
| Unresolved matter / follow-up | Missing or conflicting evidence; affected decision; next request/action; owner/date only if supplied | — |
| Approved carry-forward | Exact edited material authorised for the next activity, linked to review decision and remaining limitations; no automatic approval | — |

Suggested review states describe this document only: draft, awaiting evidence, returned for revision, held, or reviewed for the stated limited use. They do not change any application schema or establish an engagement conclusion. An action status is recorded separately from an evidence or reviewer status.

## Field definitions and fillable templates

Each table is a blank working view. Duplicate a view or its item rows only for a distinct system, test, claim or event. Keep related evidence under stable IDs.

### Vendor due-diligence questionnaire

| Field | Definition / purpose | Entry |
|---|---|---|
| Service and requirement | Exact service/tier/use, data role and criterion source relevant to the decision. | — |
| Question / vendor claim | Specific question and attributable response; unanswered questions stay open. | — |
| Current supporting evidence | Contract/configuration/test reference, date, scope and applicability; capability statement separate from deployment. | — |
| Data and dependency boundary | Context sent, retention/training/access/deletion claims and supporting evidence; fallback/exit assumptions where material. | — |
| Reviewer observation / gap | What the reviewer actually inspected and observed, with locator; distinguish vendor assertion from documentary support, unresolved question and proposed restriction. | — |
| Owner decision | Procurement/data/security/legal decision where relevant, with limits; no automatic vendor approval. | — |

**Completion guidance:** Reuse AMO-W-000002; do not turn completed answers into approval or infer the current tier from generic marketing.

### Agent permission and tool-access review

| Field | Definition / purpose | Entry |
|---|---|---|
| Principal / resource / action | Who or what can act on which record/tool and for what purpose/tenant/time. | — |
| Declared boundary | Applicable authorisation/policy and expected data/action restrictions. | — |
| Effective evidence | Configuration, context, trace or authorised test showing actual access; note absent evidence. | — |
| Delegation / revocation | Identity chain, privilege changes and expiry/withdrawal evidence where relevant. | — |
| Difference and consequence | Excess/missing access or uncertainty and the affected use; do not claim a test was run. | — |
| Review and follow-up | Owner action, authorised retest and reviewer restriction or decision. | — |

**Completion guidance:** For a RAG-only system, assess retrieval/context access without pretending it has autonomous action tools. An output refusal is not proof of source access enforcement.

### Human override/control review

| Field | Definition / purpose | Entry |
|---|---|---|
| Trigger and authorised operator | When intervention is required and who can decide/act. | — |
| Detection and decision evidence | What the operator could see and when; authority and response constraints. | — |
| Intervention / acknowledgement | Requested action and receipt, with timestamps/source IDs. | — |
| Downstream effect | Evidence generation/action actually stopped or was contained; distinguish receipt from effect. | — |
| Recovery and residual exposure | What remained queued/delivered and who may resume; unknowns stay open. | — |
| Reviewer conclusion | Bounded conclusion, unperformed checks and further containment evidence needed. | — |

**Completion guidance:** Trace detection → authority → action → effect → recovery. A visible button or acknowledgement alone cannot complete the effect field.

### AI monitoring review

| Field | Definition / purpose | Entry |
|---|---|---|
| Purpose / signal / baseline | What risk a signal detects, exact system/population and comparison baseline. | — |
| Definition and coverage | Numerator/denominator, slices, collection process and missing/duplicate records; where logging supports the claim, record joinable request/baseline IDs, timestamp basis and evidence-access limits without copying unnecessary payloads. | — |
| Threshold and authority | Supplied trigger/rationale and decision owner; missing thresholds are not invented. | — |
| Actual observation and response | Observed period/value, alert, triage and action evidence separately. | — |
| Drift / limits / feedback | Changed source/model/population, comparability limits and whether feedback enters evaluation. | — |
| Reviewer and next check | Supported monitoring claim, gap, accountable action and review decision. | — |

**Completion guidance:** Reuse AMO-W-000013/AMO-W-000016/AMO-W-000021 components. Satisfaction, drift and factual correctness are different constructs; a dashboard label is not a tested control.

### AI incident review

| Field | Definition / purpose | Entry |
|---|---|---|
| Event identity / evidence type | Actual incident, reported event, near miss or exercise; maintain distinction. | — |
| Chronology and scope | Attributable timestamps, affected boundary/records and unknown extent. | — |
| Assertions versus observations | Who reported what and the evidence actually available. | — |
| Containment / preservation | Actions supported by records, authority and evidence preserved; proposed actions separate. | — |
| Cause hypotheses / recovery | Plausible explanations versus corroborated cause; authorised recovery conditions. | — |
| Escalation / notifications | Supplied process and responsible specialist decisions; no invented legal duty or completed notification. | — |
| Open matters and review | Outstanding scope/testing/evidence and human review of the bounded incident record. | — |

**Completion guidance:** An exercise result is not production incident evidence. Do not delete logs, contact third parties or investigate beyond authority because a worksheet suggests a question.

### AI change-management review

| Field | Definition / purpose | Entry |
|---|---|---|
| Change and approved baseline | Purpose, source/model/template/data/permission delta and intended baseline. | — |
| Effective state evidence | Actual deployed/corpus/configuration identity and comparison; no access means not inspected. | — |
| Acceptance and test evidence | Source-based criteria, relevant positive/negative cases, actual results and exclusions. | — |
| Release authority and separation | Who may approve/release, recorded decisions and any conflicts. | — |
| Rollback / fallback | Plan, dependency/semantic limits and actual exercise evidence if supplied. | — |
| Post-change monitoring / retest | Operating period, regression/entitlement results and unresolved consequences. | — |
| Reviewer disposition | Scoped readiness or hold recommendation with decision owner; ticket closure is not effectiveness. | — |

**Completion guidance:** Reuse AMO-W-000021 and its existing exact sources. Preserve current/previous baseline differences; no plan, ticket or resource title authorises production release.

## Bundle completion guidance

Choose modules by the objective; share evidence IDs across them. Keep supplier statements, written design, actual configuration and observed results in separate cells. Use specialist sign-off only when actually obtained. For relevant logs, examine supplied access, integrity, minimisation and retention requirements and actual evidence; absent evidence is not proof that logging is insecure. Do not prescribe a retention duration. Apply the organisation's approved retention and privacy requirements. A fallback provider requires its own capability/data-boundary evidence, not an assumed equivalent.

Before carrying material forward, reconcile the question, criterion, source and actual observation. Preserve original evidence in the institution's authorised repository. Use only permitted excerpts in an AI surface; do not paste private credentials or unapproved identifiers. Local saving is not evidence custody or permission to disclose to a provider. A proposed test or a completed form cannot stand in for a recorded result.

The audit supervisor decides sufficiency and reliance; the specialist decides matters within their competence and authority. Management validates its own representations and owns corrective action. Management agreement is not the audit reviewer's independent closure decision. Carry-forward remains editable and requires explicit review.

## Example completed synthetic excerpt

| Module | Synthetic completed entry |
|---|---|
| Vendor | CR-08 describes training-use/filter capabilities; applicable tier, terms and retention evidence absent; no approval concluded |
| Permission | CR-03 prohibits staff context for PUBLIC; CR-05 Q04 includes K-OPS v1 in public context; wider extent unknown |
| Override | CR-07 acknowledgement 10:07:02 and answer marker 10:07:05; delivery/buffering unknown; containment not established |
| Monitoring | CR-09: 19/20 voluntary ratings helpful; total answers unknown; cannot infer 95% accuracy |
| Incident | CR-11 remains awaiting evidence; no actual personal-data record supplied; preserve scope uncertainty |
| Change | CR-10 claims index updated 26 September; CR-04 still names 24 September C-17; effective state needs reconciliation |

Evidence obtained date in the case: 7 October. Proposed reviewer decision: hold the affected control claims pending specific evidence; no actual system change, notification or closure is performed.

This is an illustrative entry from the bundled fictional case, not work performed on a real organisation. Review dates/decisions shown as proposed must not be represented as real sign-offs.

## Limitations

- Does not supply a complete sector programme, infrastructure inspection or legal determination.
- Modules must not turn vendor or management assertions into independent operational evidence.
- Authorised real tests and source records are required for conclusions about actual systems.
- This is a supplied-evidence review aid. It does not perform an audit, issue an audit opinion, establish control effectiveness, certify anything, determine legal compliance or grant professional approval. The authorised audit professional retains judgement and reliance decisions. Use only information permitted for the selected provider; local encrypted storage does not itself authorise provider disclosure.

## Related Amo.ng resources

**Prompts:** [AMO-P-000074 — Evidence-Grounded Shadow AI Risk Assessment and Governance Plan](https://amo.ng/prompts/shadow-ai-risk-assessment-business) v2.0.0; [AMO-P-000242 — RAG Retrieval and Citation Quality Evaluation Lab](https://amo.ng/prompts/rag-retrieval-citation-quality-evaluation-lab) v1.0.0; [AMO-P-000071 — Evidence-Grounded AI Output Verification and Release Gate](https://amo.ng/prompts/ai-output-verification-checklist) v2.0.0

**Workflows:** [AMO-W-000002 — Vendor Procurement Due Diligence](https://amo.ng/workflows/vendor-procurement-due-diligence); [AMO-W-000013 — Review Production AI Evaluation Reliability](https://amo.ng/workflows/production-ai-evaluation-reliability-review); [AMO-W-000016 — Assess Enterprise Knowledge and RAG Readiness](https://amo.ng/workflows/enterprise-knowledge-and-rag-readiness-review); [AMO-W-000021 — Assure an AI System Change for Production Release](https://amo.ng/workflows/ai-system-change-production-release-assurance)

**Skills:** [AMO-S-000007 — Diagnose RAG Grounding and Retrieval Failures](https://amo.ng/skills/diagnose-rag-grounding-and-retrieval-failures); [AMO-S-000012 — Assess AI Evaluation Evidence for Decision Use](https://amo.ng/skills/assess-ai-evaluation-evidence-for-decision-use); [AMO-S-000016 — Trace Consequential Claims to Original Evidence](https://amo.ng/skills/trace-consequential-claims-to-original-evidence)

These are complementary methods, not additional proof of the same claim. Read their current source boundaries; open their normal resource pages for the full instructions. Existing resources and their verification status remain unchanged.

## Related audit Prompts and Workflow

- [AMO-P-000361](https://amo.ng/prompts/challenge-internal-audit-engagement-scope-evidence-plan) bounds the engagement.
- [AMO-P-000363](https://amo.ng/prompts/challenge-audit-finding-management-response-closure-claim) challenges management response/closure; [AMO-W-000044](https://amo.ng/workflows/evidence-controlled-internal-audit-working-paper) records reviewed analysis.

Use of these resources is subject to the [Amo.ng Terms](https://amo.ng/terms).