Evidence-Grounded Agentic RAG Architecture and Acceptance Plan

Develop an evidence-grounded RAG architecture and acceptance plan for the system described below. Treat agentic RAG as an option to justify, not a predetermined answer.

Project inputs

Blocking inputs for a reliable recommendation:
- Project context: [Project context]
- Assistant purpose and target users: [Assistant purpose and target users]
- Knowledge source inventory: [Knowledge source inventory]
- Risk and compliance constraints: [Risk and compliance constraints]
- Definition of done: [Definition of done]

Useful supporting inputs:
- Representative source samples: [Representative source samples]
- Query and answer examples: [Query and answer examples]
- Tool and system constraints: [Tool and system constraints]
- Citation and provenance requirements: [Citation and provenance requirements]
- Human review and authority rules: [Human review and authority rules]
- Nonfunctional requirements: [Nonfunctional requirements]
- Evaluation evidence: [Evaluation evidence]

Input and evidence discipline

1. Begin with an input sufficiency check. Identify missing, ambiguous, stale, or conflicting information and separate blocking gaps from non-blocking gaps.
2. Ask concise clarification questions for gaps that could materially change source authorization, privacy controls, architecture selection, escalation rules, or acceptance criteria. If an answer is unavailable, continue only with a bounded draft where safe; preserve the item as an unknown and explain its design impact.
3. Label substantive statements as one of: supplied fact, observed in supplied material, assumption, recommendation, unresolved conflict, unknown, or reported execution evidence.
4. Do not invent document contents, source coverage, system access, benchmark results, legal requirements, stakeholder approval, or production behavior.
5. When sources conflict, record the conflict rather than silently choosing one. Recommend a precedence rule using factors such as source owner, authority, jurisdiction, version, effective date, approval state, and freshness.

ChatGPT operating boundary

Use ChatGPT to inspect text, files, schemas, logs, test results, and diagrams actually supplied in this conversation and to reason about architecture choices. If browsing, code execution, connectors, or other tools are available, use them only when explicitly enabled and report what was actually inspected or run. Cite or identify the resulting evidence.

Do not imply that ChatGPT accessed an internal repository, vector database, production environment, identity system, ticketing system, or monitoring platform unless corresponding access and results are present in the conversation. Do not modify data, create indexes, call production APIs, deploy components, approve the design, or contact users. Produce a design and test plan only. Any implementation, destructive index operation, permission change, production test, deployment, or approval requires an authorized human and the organization’s change controls.

Architecture analysis

1. Define the assistant’s answerable scope, excluded scope, user groups, trust boundaries, data classifications, and likely query classes. Identify whether answers require document retrieval, structured account data, deterministic computation, transactional tools, or human judgment.
2. Compare basic RAG, enhanced RAG, and agentic RAG using a decision matrix. Evaluate retrieval complexity, multi-step reasoning need, tool-use need, predictability, attack surface, latency, cost, observability, maintainability, evaluation burden, and failure containment.
3. Recommend the least complex approach that satisfies the supplied requirements. If agentic behavior is justified, specify exactly which decisions require an agent, which remain deterministic, and the limits on iterations, retrieval calls, tool calls, time, and cost. If it is not justified, recommend basic or enhanced RAG without forcing an agentic design.
4. Separate the offline knowledge pipeline from the online answer pipeline.

Offline knowledge pipeline

Design a governed flow for source registration, authorization, extraction, normalization, deduplication, structure preservation, chunking, metadata enrichment, embedding, lexical indexing, validation, publication, monitoring, re-indexing, and retirement.

For each source class, address:
- owner, authority, permitted audiences, tenant or account boundary, jurisdiction, effective date, expiry, version, freshness target, and deletion obligations;
- parsing and structure preservation for headings, tables, lists, FAQs, PDFs, scanned documents, policy clauses, attachments, and long-form material;
- chunk boundaries based on semantic units and document hierarchy rather than arbitrary fixed lengths alone;
- candidate chunk-size and overlap ranges as hypotheses to evaluate, not universal facts;
- parent-child retrieval, table serialization, adjacent-context expansion, and links back to the original passage;
- stable document, version, section, and chunk identifiers needed for citations and deletion propagation;
- access-control metadata enforced before or during retrieval, not merely filtered after generation;
- quarantine and recovery behavior for malformed, untrusted, stale, unauthorized, or partially indexed content;
- versioned index releases, validation gates, rollback to the last accepted index, and proof that deletions propagate to lexical, vector, cache, and derived stores.

Online answer pipeline

Provide a numbered text diagram and a stage contract covering:
- request authentication and tenant resolution;
- input validation, prompt-injection screening, and sensitive-data handling;
- query classification and answerability screening;
- deterministic policy routing;
- query rewriting or decomposition when justified;
- retrieval planning and source selection;
- metadata and access-control filtering;
- lexical, semantic, structured-data, or hybrid retrieval;
- fusion, deduplication, reranking, and context-budget allocation;
- freshness, authority, contradiction, and coverage checks;
- context assembly with provenance retained;
- grounded answer generation;
- claim extraction and claim-to-source support checking;
- citation validation;
- calibrated decision signals for answer, clarify, abstain, or escalate;
- response policy enforcement and delivery;
- privacy-aware logging, feedback capture, and monitoring.

For every stage, state its purpose, inputs, outputs, decision rule, permitted tools, evidence retained, timeout or budget, principal failure modes, fallback path, and any human gate. Keep retrieval, generation, verification, and response authorization distinct.

Retrieval and grounding design

1. Define source-priority rules without relying on a single source. Explain how authority, access rights, effective dates, freshness, query relevance, and source quality interact.
2. Recommend lexical, semantic, hybrid, metadata-filtered, graph, or structured retrieval only where justified. Specify fusion and reranking behavior, candidate-set limits, and how retrieval diversity is preserved.
3. Address entity ambiguity, acronyms, multilingual queries, temporal questions, near-duplicate passages, superseded policies, empty filters, and questions spanning multiple documents.
4. For structured or account-specific data, prefer authorized deterministic APIs or database services over embedding volatile records. Define read-only versus state-changing tools, schema validation, least-privilege credentials, timeouts, retries, idempotency where relevant, and safe error handling.
5. Treat retrieved text and tool output as untrusted input. Prevent documents from overriding system policy, requesting secrets, changing tool permissions, or directing unauthorized actions.
6. Define context inclusion and exclusion rules. Explain how the system detects insufficient evidence, conflicting evidence, and excessive context dilution.

Citation and claim-verification protocol

Define:
- which factual claims require citations and which conversational statements do not;
- the minimum citation unit and required provenance fields;
- how a citation must resolve to the exact source version and supporting passage;
- checks for citation existence, access authorization, passage relevance, entailment, freshness, and source authority;
- claim-level coverage expectations and handling of partially supported sentences;
- treatment of derived answers that combine multiple sources or deterministic calculations;
- behavior for inaccessible, missing, stale, superseded, or contradictory evidence;
- a rule that unsupported claims are removed, qualified, converted into a clarification request, or refused rather than assigned a decorative citation.

Confidence, fallback, and human control

Do not present an uncalibrated model confidence score as a probability of correctness. Define observable decision signals such as retrieval coverage, reranker separation, source authority, claim support, contradiction status, citation validity, tool success, and query-class risk. Explain how thresholds would be calibrated on representative labeled queries.

Create a fallback and escalation matrix for no results, weak results, conflicting sources, stale sources, unauthorized sources, unsupported requests, ambiguous identity or tenant, tool timeout, malformed tool output, prompt injection, low evidence coverage, and sensitive high-impact requests. For each condition specify detection evidence, safe user response, retry allowance, logging, escalation destination, and prohibited behavior.

Place human review gates only where they change risk. At minimum, evaluate gates for legal, medical, financial, compliance, billing, privacy, security, employment, and consequential account actions. State who may review or approve, what evidence they receive, the service-level expectation, and what happens if no reviewer is available. The generated answer must never represent human approval that has not occurred.

Safety and operational controls

Address tenant isolation, document-level authorization, personally identifiable information, secrets, retention, redaction, encryption assumptions, audit logs, regional restrictions, abuse monitoring, rate limits, denial-of-wallet risks, tool-loop limits, cache leakage, poisoning of the knowledge corpus, data deletion, incident containment, and rollback. Identify stop conditions that should block launch or suspend responses.

Evaluation and acceptance design

Define representative evaluation sets segmented by query class, source class, user authorization, language if applicable, freshness, ambiguity, contradiction, answerable versus unanswerable requests, injection attempts, and risk tier. Prevent test-set leakage and distinguish offline evaluation, adversarial testing, shadow testing, limited rollout, and production monitoring.

Include formulas or precise definitions where applicable for:
- retrieval precision at k, recall at k, mean reciprocal rank or normalized discounted cumulative gain;
- authorized-retrieval rate and cross-tenant leakage rate;
- claim-level citation coverage, citation correctness, and source-version resolution;
- groundedness or faithfulness and contradiction rate;
- answerability classification, abstention precision and recall, and escalation accuracy;
- task success, reviewer agreement, and material-error severity;
- p50 and p95 latency, timeout rate, tool failure rate, and cost per resolved query;
- freshness compliance, index failure rate, deletion-propagation compliance, and rollback success.

For every proposed acceptance test, provide: test ID, requirement or risk, test fixture, procedure, expected observation, actual observation, evidence reference, threshold, status, owner, and remediation. Use not run for actual observation and unverified for status unless supplied evidence shows that the test was executed. If evaluation evidence is supplied, distinguish reported results from results directly inspected or reproduced in the available tool context.

Do not invent universal thresholds. Derive thresholds from supplied service requirements, risk tolerance, baseline performance, and labeled evaluation data. Where these are missing, provide clearly labeled provisional targets and the calibration work required before approval.

Required deliverable

Return one structured architecture and acceptance document containing:

1. Input Sufficiency and Evidence Ledger
A table of each required input, availability, evidence location, reliability, conflicts, assumptions, blocking status, and clarification needed.

2. Scope, Trust Boundaries, and Answerability Contract
Supported users and query classes, excluded uses, data classifications, system boundaries, and answer, clarify, abstain, or escalate rules.

3. Architecture Decision Record
A scored comparison of basic, enhanced, and agentic RAG; decisive trade-offs; recommended pattern; rejected alternatives; assumptions; and conditions that would reverse the decision.

4. Component and Data-Flow Design
A text architecture diagram plus component contracts showing data stores, models, retrievers, rerankers, policy services, tools, verification services, logs, and human-review interfaces.

5. Offline Corpus Governance and Indexing Specification
Source registry, parsing and chunking rules by source type, metadata schema, authorization enforcement, versioning, freshness, index validation, deletion propagation, release, and rollback.

6. Online Workflow Stage Contract
A stage-by-stage table with purpose, inputs, outputs, decisions, tools, evidence, budgets, failure paths, and human gates.

7. Retrieval and Context Assembly Specification
Query routing, candidate generation, filters, fusion, reranking, context allocation, contradiction handling, and representative tuning hypotheses.

8. Tool and Authority Register
For each API, database, connector, or service: purpose, data accessed, read or write authority, authentication, schema constraints, side effects, timeout, retry policy, audit evidence, failure behavior, and required human authorization.

9. Citation and Claim-Support Specification
Citation schema, claim-support rules, validation sequence, conflict handling, unsupported-claim behavior, and examples grounded only in supplied materials.

10. Decision-Signal, Fallback, and Escalation Matrix
Signals, calibration requirements, thresholds or provisional targets, response behavior, retry limits, escalation owners, and prohibited actions.

11. Security, Privacy, and Abuse-Control Register
Threat or hazard, affected boundary, likelihood and impact rationale, preventive control, detective control, response, residual risk, owner, and launch-blocking status.

12. Evaluation and Acceptance Matrix
Metric definitions, dataset slices, test cases, expected observations, actual observations, evidence references, thresholds, statuses, and remediation owners.

13. Phased Implementation and Rollout Plan
Discovery, prototype, offline evaluation, security review, shadow mode, limited rollout, production readiness, monitoring, rollback triggers, dependencies, approval gates, and exit criteria. Keep all phases marked proposed unless execution evidence is supplied.

14. Open Decisions and Handoff Register
Decision, options, recommendation, accountable owner, required evidence, deadline if supplied, dependency, and current state.

15. Final Recommendation and Readiness State
Summarize the recommended architecture, highest residual risks, unresolved blockers, next authorized action, and one readiness state: design draft, awaiting inputs, ready for technical review, ready for controlled implementation, or not recommended. Do not use approved, tested, verified, deployed, production-ready, or completed unless the conversation contains matching evidence and authority.

Final verification before responding

- Reconcile every requirement and material risk to at least one architecture control and one evaluation or review method.
- Confirm that every online stage has defined inputs, outputs, evidence, failure handling, and authority boundaries.
- Trace representative answer claims from query through authorized retrieval, source version, supporting passage, verification result, and final citation.
- Confirm access controls are applied before protected content can enter model context.
- Confirm unsupported, conflicting, stale, unauthorized, and high-risk cases have testable fallback behavior.
- Confirm agent loops and tool use have explicit budgets, stop conditions, and audit evidence.
- Confirm acceptance rows contain expected and actual observations and that unexecuted tests remain marked not run and unverified.
- Reconcile assumptions, open conflicts, residual risks, and blockers with the final readiness state.
- Return only the requested design document; do not claim implementation or evaluation work occurred when it did not.