quality-agent-spec

Agent Specification: Quality Agent

1. Agent Overview

| Attribute | Value | |-----------|-------| | **Name** | Quality Agent | | **ID** | research-quality-agent | | **Purpose** | Assess source quality using GRADE framework, validate FAIR compliance, generate quality reports, and enforce quality gates | | **Lifecycle Stage** | Cross-cutting (integrated with Documentation, Integration stages) | | **Model** | sonnet | | **Version** | 1.0.0 | | **Status** | Draft |

Description

The Quality Agent ensures research integrity by systematically evaluating sources using established frameworks. It calculates multi-dimensional quality scores (authority, currency, accuracy, coverage, objectivity), applies GRADE methodology for evidence assessment, validates FAIR principles for data quality, and generates actionable quality reports with recommendations. The agent serves as a quality gate, flagging low-quality sources and recommending alternatives.

2. Capabilities

Primary Capabilities

| Capability | Description | NFR Reference | |------------|-------------|---------------| | Multi-Dimensional Scoring | Evaluate authority, currency, accuracy, coverage, objectivity | BR-QA-002 | | GRADE Assessment | Rate evidence quality (High/Moderate/Low/Very Low) | NFR-QA-04 | | FAIR Validation | Check Findable, Accessible, Interoperable, Reusable principles | NFR-QA-04 | | Quality Reporting | Generate clear, actionable quality reports | NFR-QA-05 | | Quality Gates | Flag low-quality sources, recommend alternatives | BR-QA-001 | | Batch Assessment | Assess multiple sources efficiently | NFR-QA-02 |

Secondary Capabilities

| Capability | Description | |------------|-------------| | Citation Analysis | Retrieve and analyze citation counts | | Venue Ranking | Assess publication venue quality (A*/A/B/C) | | Conflict Detection | Identify contradictory or conflicting quality signals | | Remediation Guidance | Provide steps to improve FAIR compliance |

3. Tools

Required Tools

| Tool | Purpose | Permission | |------|---------|------------| | Bash | Execute external APIs, file operations | Execute | | Read | Access source metadata, existing assessments | Read | | Write | Save quality reports, update metadata | Write | | Glob | Find sources for batch assessment | Read | | Grep | Search metadata for quality indicators | Read |

External APIs

| API | Endpoint | Purpose | Auth | |-----|----------|---------|------| | Semantic Scholar | `api.semanticscholar.org` | Citation counts | None | | CrossRef | `api.crossref.org` | Publication metadata | None | | OpenCitations | `opencitations.net/index` | Citation network | None |

4. Triggers

Automatic Triggers

| Trigger | Condition | Action | |---------|-----------|--------| | Documentation Complete | Paper documented (UC-RF-003) | Assess quality | | Acquisition Complete | Paper acquired (UC-RF-002) | FAIR pre-check | | Workflow Stage | UC-RF-008 initiates quality stage | Batch assessment |

Manual Triggers

| Trigger | Command | Description | |---------|---------|-------------| | Single Assessment | `aiwg research assess-quality REF-XXX` | Assess one source | | Batch Assessment | `aiwg research assess-quality --all` | Assess all sources | | FAIR Check | `aiwg research fair-check REF-XXX` | FAIR validation only | | Quality Report | `aiwg research quality-report` | Generate summary report |

5. Inputs/Outputs

Inputs

| Input | Format | Source | Validation | |-------|--------|--------|------------| | Source Metadata | YAML frontmatter + Markdown | `.aiwg/research/sources/` | Required fields present | | Citation Information | Structured citation | Source metadata | Valid format | | Source Type | Enum | Source metadata | Valid type | | Publication Date | ISO 8601 | Source metadata | Valid date |

Outputs

| Output | Format | Location | Retention | |--------|--------|----------|-----------| | Quality Report | Markdown | `.aiwg/research/quality/{REF-XXX}-quality-report.md` | Permanent | | Quality Score | Integer (0-100) | Source metadata frontmatter | Permanent | | GRADE Rating | Enum | Source metadata frontmatter | Permanent | | FAIR Compliance | Boolean + details | Source metadata frontmatter | Permanent | | Aggregate Report | Markdown | `.aiwg/research/quality/quality-summary.md` | Updated continuously |

Output Schema: Quality Report Frontmatter

---
ref_id: REF-025
assessment_date: 2026-01-25
quality_score: 87
grade_rating: High
fair_compliant: true
dimension_scores:
  authority: 85
  currency: 90
  accuracy: 95
  coverage: 80
  objectivity: 85
grade_details:
  study_design: "Randomized controlled trial"
  risk_of_bias: "Low"
  consistency: "Consistent with other studies"
  directness: "Directly addresses question"
  precision: "Adequate sample size"
  publication_bias: "No evidence"
fair_details:
  findable: true
  accessible: true
  interoperable: true
  reusable: true
recommendation: "Approved for integration"
---

Output Schema: Quality Report Body

# Quality Assessment Report: REF-025

## Executive Summary

**Overall Score:** 87/100 (High Quality)
**GRADE Rating:** High (strong confidence in evidence)
**FAIR Compliance:** 4/4 principles met
**Recommendation:** Approved for integration

## Dimension Scores

| Dimension | Score | Weight | Weighted |
|-----------|-------|--------|----------|
| Authority | 85 | 30% | 25.5 |
| Currency | 90 | 20% | 18.0 |
| Accuracy | 95 | 25% | 23.75 |
| Coverage | 80 | 15% | 12.0 |
| Objectivity | 85 | 10% | 8.5 |
| **Total** | - | 100% | **87.75** |

## Strengths

- Peer-reviewed in A* venue (ACM CCS)
- Recent publication (2023)
- Comprehensive methodology documented
- Large sample size (10,000 users)

## Limitations

- Single institution study (generalizability concern)

## Recommendations

- Approved for integration
- Suitable for primary evidence
- Consider supplementing with multi-site studies

6. Dependencies

Agent Dependencies

| Agent | Re