/citation-audit

name: citation-audit
description: 'Verify that every citation in a manuscript is real, correctly attributed, and accurately described. Detects ghost papers, wrong arXiv IDs, inverted claims, and dead links by fetching each cited work. Optional fix mode applies bib metadata corrections and surfaces prose rewrites for claim errors. Triggers on: "check my citations", "verify references", "citation audit", "are my references real", "check bib", "reference check", "bib audit", "citation verification". Companion to manuscript-review (Pass 5 hygiene); this skill audits factual truth.'
metadata:
  version: 1.0.0
  complements: [manuscript-review, research-critique, literature-review]
  category: review
  tags: [citations, bibliography, fact-check, hallucination, references]
  difficulty: advanced
  phase: review

Citation Audit Skill

Purpose

Verify every citation in a manuscript against its actual source. LLMs hallucinate citations, invent arXiv IDs, misattribute findings, and confuse authors. This skill catches all of that by fetching and reading each cited work.

Why This Exists

LLMs are unreliable with citations in three distinct ways: 1. **Ghost papers** — The paper does not exist. Title, authors, or venue are fabricated. 2. **Wrong metadata** — The paper exists but the bib entry has the wrong arXiv ID, wrong authors, wrong year, or wrong venue. 3. **Inverted claims** — The paper exists and the bib is correct, but the manuscript mischaracterizes what the paper says.

All three are invisible to structural audits (cross-reference checks, compilation tests). They require reading the actual cited work.

Inputs

• The manuscript `.tex` file(s)
• The `.bib` file
• Web access (to fetch papers from arXiv, conference sites, URLs)

Execution

Phase 1: Extract citation contexts

For each `\citep{}`, `\citet{}`, `\cite{}` in the manuscript: 1. Record the bib key 2. Record the surrounding sentence or paragraph (the **claim context**) 3. Classify the claim type:

• **FACTUAL**: "X et al. found Y" / "X et al. measured Y"
• **METHODOLOGICAL**: "We follow X" / "We use the benchmark from X"
• **POSITIONAL**: "Unlike X, we..." / "X does not measure..."
• **PARENTHETICAL**: "(X, 2024)" — no specific claim, just a reference

4. For FACTUAL and POSITIONAL claims, extract the specific assertion the manuscript makes about the cited work

Phase 2: Verify bib entry metadata

For each bib entry, verify against the actual source:

**For arXiv papers (`eprint` field present):** 1. Fetch `https://arxiv.org/abs/{eprint_id}` 2. Compare: title, authors, year 3. If the fetched paper has a DIFFERENT title/authors than the bib entry, this is a **WRONG ID** or **GHOST PAPER**

**For conference/journal papers (`booktitle` or `journal` field):** 1. Search for the paper by title + author on the web 2. Verify: venue, year, author list 3. If the paper cannot be found at the stated venue, flag as **UNVERIFIABLE** or **GHOST PAPER**

**For web resources (`howpublished` with URL):** 1. Fetch the URL 2. Verify it loads and the content matches the described resource 3. If the URL is dead or redirects to unrelated content, flag as **DEAD LINK**

**For each entry, check:**

• [ ] Paper exists (reachable via arXiv, DOI, URL, or web search)
• [ ] Title matches (exact or near-exact)
• [ ] Authors match (at least first author correct)
• [ ] Year matches
• [ ] Venue matches (if applicable)
• [ ] Entry type appropriate (`@inproceedings` for conferences,

`@article` for journals, `@misc` for preprints/blogs)

Phase 3: Verify claim accuracy

For each FACTUAL or POSITIONAL claim: 1. **Read the cited paper** (abstract + relevant sections at minimum) 2. Compare the manuscript's claim against what the paper actually says 3. Classify:

• **ACCURATE** — The claim faithfully represents the cited work
• **INACCURATE** — The claim mischaracterizes the cited work
• **INVERTED** — The claim says the opposite of what the paper found
• **OVERCLAIMED** — The claim is stronger than what the paper supports
• **UNDERCLAIMED** — The cited work supports a stronger claim than stated
• **UNVERIFIABLE** — Cannot access the paper to verify

For INACCURATE and INVERTED findings, provide:

• What the manuscript claims
• What the cited paper actually says
• The specific section/page of the cited paper that contradicts the claim
• A suggested correction

Phase 4: Check for missing citations

Scan the manuscript for: 1. Claims that cite no source but should (empirical claims without attribution) 2. Tools, benchmarks, or datasets mentioned by name without citation 3. Methods described as "standard" or "well-known" that have a canonical citation

Output Format

Per-citation report

### [bib_key] — [VERDICT]

**Bib entry:** [title] by [authors] ([year])
**Actual paper:** [actual title] by [actual authors] ([actual year])
**Metadata match:** title [✓/✗] | authors [✓/✗] | year [✓/✗] | venue [✓/✗]

**Claim in manuscript (line N):** "[exact text]"
**What the paper actually says:** "[summary of actual finding]"
**Claim accuracy:** [ACCURATE / INACCURATE / INVERTED / OVERCLAIMED / UNDERCLAIMED]

**Fix required:** [description of what needs to change, or "None"]

Summary table

| Bib Key | Exists | Metadata | Claim | Verdict |
|---------|--------|----------|-------|---------|
| key1    | ✓      | ✓        | ✓     | PASS    |
| key2    | ✓      | ✗        | ✗     | FAIL    |
| key3    | ✗      | —        | —     | GHOST   |

Verdict categories

• **PASS** — Paper exists, metadata correct, claims accurate
• **METADATA** — Paper exists, bib entry has errors (wrong ID, wrong

authors, wrong year)

• **CLAIM** — Paper exists, metadata correct, but manuscript

mischaracterizes it

• **GHOST** — Paper does not exist as described
• **DEAD** — URL/link is broken
• **UNVERIFIABLE** — Cannot access the paper to verify

Severity

• **CRITICAL**: GHOST papers, INVERTE