/evidence-based-rag

description: Evidence-first retrieval-augmented reasoning skill for decision-critical scenarios. Retrieves information from a specified knowledge base, extracts verifiable evidence, detects conflicting claims, and evaluates answer sufficiency with explicit confidence and risk signals. Produces traceable outputs suitable for agent-level decision control and escalation.

Evidence-Based RAG

Overview

`evidence_based_rag` is an **evidence-first retrieval-augmented reasoning skill** designed to retrieve, extract, and organize factual information from a specified knowledge base in support of **decision-critical scenarios** (e.g., due diligence, compliance review, research analysis).

The responsibility of this skill is to produce **traceable, verifiable, and auditable findings**, while explicitly surfacing conflicts and uncertainty signals. It does **not** perform business judgments or replace human decision-making.

---

Skill Responsibility

This skill is responsible for:

• Retrieving relevant information from a specified knowledge base
• Transforming raw text into verifiable evidence units
• Detecting conflicting claims related to the same factual question
• Evaluating answer sufficiency and confidence
• Explicitly surfacing uncertainty and risk signals, along with suggested follow-up actions

This skill is **not responsible for**:

• Arbitrating between conflicting claims
• Making investment, legal, or business decisions
• Controlling workflow execution (e.g., retries, looping, escalation)

---

Design Invariants

The following invariants must hold in any execution environment:

1. **Evidence-bound**

• All key conclusions must be supported by evidence from the knowledge base
• No external knowledge or common-sense assumptions may be introduced

2. **Traceability**

• Every material claim must be traceable to a specific location in the original source

(document name, page, section, paragraph, line, or clause)

3. **Non-assumptive**

• Insufficient or conflicting evidence must be explicitly marked as uncertain
• No gap-filling based on plausibility, convention, or experience

4. **Conflict-aware**

• Multiple conflicting claims about the same fact must be surfaced with their citations
• Conflicts are treated as risk signals, not automatically resolved

5. **Agentic-compatible**

• Outputs must expose structured signals usable by downstream agents or humans:

`confidence`, `sufficient`, `conflicts`, `suggested_next_actions`

6. **Subject Existence** (Hard Gate)

• A conclusion MUST NOT be produced unless at least one candidate subject that explicitly matches the query subject class is identified in the knowledge base
• If no such subject is found, the system MUST return `sufficient = false` and MUST NOT substitute semantically similar subjects
• "I don't know" is a valid capability in agent systems; "hard answering" under subject absence is a defect
• This invariant prevents semantic fallback under subject absence
• Applies universally to all subject types: entities, individuals, products, locations, time periods, jurisdictions, etc.

7. **Entity Binding**

• A candidate subject MUST have an explicit and verifiable binding to the parent entity specified in the query subject class
• Role or title similarity alone is insufficient
• Valid binding mechanisms: ownership, registration, contractual role, or explicit statement in source text
• If the parent entity relationship cannot be explicitly established from the knowledge base, the subject MUST NOT be considered a valid candidate
• This invariant prevents accepting subjects that "look like" matches but lack the required entity relationship

8. **Subject-Evidence Integrity**

• Any evidence used to support a conclusion MUST explicitly confirm that the subject of the evidence matches the subject of the query
• If subject identity or scope cannot be unambiguously established, the evidence MUST NOT be used to support the conclusion
• Subject drift is a critical error that invalidates the entire conclusion
• This invariant applies universally: companies, individuals, products, locations, time periods, jurisdictions, legal entities, or any other subject type

---

When to Use

Use this skill when the task requires:

• **Knowledge base Q&A** - Answering questions based on specific documents or knowledge bases with source attribution
• **Evidence verification** - Retrieving and verifying factual claims from documents
• **Due diligence queries** - Investigating facts from multiple sources with conflict detection
• **Multi-source analysis** - Synthesizing information from multiple documents with explicit citations
• **Fact-checking** - Verifying claims against source documents

Typical use cases include:

• Knowledge base queries requiring evidence traceability
• Document analysis with source attribution
• Compliance or legal document review
• Research analysis and fact verification

Not suitable for:

• Casual or conversational generation tasks
• Queries that do not require source attribution or traceability
• Creative writing or brainstorming tasks

---

Inputs & Outputs

This skill does **not** impose a fixed schema on inputs or outputs.

• Inputs are organized by the calling Agent or runtime (e.g., question, knowledge base scope, constraints).
• Outputs aim to be **structured and machine-consumable**, but concrete field shapes are determined by the integration context.

Core invariants enforced by the skill:

• All conclusions must be evidence-backed
• Conflicts, uncertainty, and risk signals must be explicitly surfaced

---

Execution State Machine

This skill implements a **strict, irreversible state machine** to eliminate non-deterministic execution paths.

States

stateDiagram-v2
    [*] --> INIT
    INIT --> SUBJECT_CLASS_IDENTIFIED
    SUBJECT_CLASS_IDENTIFIED --> CANDIDATE_SUBJECT_DISCOVERED
    CANDIDATE_SUBJECT_DISCOVERED