/publication-trust

name: publication-trust
description: Assess the scientific integrity and trustworthiness of publications before relying on their findings. Use this skill whenever evaluating a paper for a workflow, citing a study, building an analysis on published methods, or when a user asks about the reliability of a study. Checks for formal retractions, corrections, expressions of concern, and — critically — informal contradictions where subsequent studies failed to reproduce key findings. Integrates with PubMed, bioRxiv, and Consensus to provide a trust assessment. Use this skill for ANY publication evaluation, retraction checking, author reliability assessment, or when a user says "can I trust this paper", "is this study reliable", "has this been refuted", or "check this publication".

Publication Trust Assessment

Evaluate the scientific integrity and reliability of publications before building analyses on their findings.

When to Use

• User wants to evaluate the reliability and reproducibility of a genomics publication
• User asks about "publication quality", "reproducibility", "trust assessment", or "paper evaluation"
• User needs to check if a paper follows ENCODE data standards and best practices
• User wants to verify that cited datasets, tools, and methods meet community standards
• Example queries: "is this paper's ChIP-seq analysis trustworthy?", "evaluate the methods in this genomics paper", "check if this study follows ENCODE standards"

Why This Matters

Not all published findings are reliable. Problems range from formal retractions (data fabrication, image manipulation) to informal contradictions where independent groups fail to reproduce key claims. In genomics and computational biology, building pipelines or analyses on unreliable findings wastes resources and propagates errors.

**The problem is not always obvious.** Formal retractions are rare compared to the number of problematic papers. More commonly:

• A subsequent study contradicts the key finding without triggering a retraction
• An erratum quietly corrects a critical result
• The original authors publish a "correction" that substantially changes conclusions
• Independent groups fail to replicate and publish their negative results

This skill provides a systematic approach to detecting these issues.

Trust Assessment Framework

Trust Levels

| Level | Label | Meaning | |-------|-------|---------| | 5 | **High confidence** | No issues found; replicated by independent groups | | 4 | **Standard** | No issues found; not yet independently replicated | | 3 | **Caution advised** | Minor corrections, errata, or partial contradictions exist | | 2 | **Reliability concerns** | Key findings contradicted by independent study, or expression of concern issued | | 1 | **Compromised** | Retracted, or key findings refuted with evidence of methodological problems |

**Default trust level is 4** (standard) — absence of evidence is not evidence of absence. A paper starts at "standard" and moves up with independent replication or down with identified issues.

Step 1: Gather Publication Metadata

For any publication being assessed, retrieve full metadata:

get_article_metadata(pmids=["PMID"])

Record:

• **article_types**: Check for "Retracted Publication", "Published Erratum", "Expression of Concern"
• **authors**: Note corresponding author and senior author (last position)
• **journal**: Impact factor context (high-profile journals ≠ automatic reliability)
• **publication_date**: Older papers have had more time for replication/refutation

If starting from a DOI:

convert_article_ids(ids=["DOI"], id_type="doi")

Step 2: Check Formal Integrity Markers

2a. Retraction Status

Search PubMed for retraction notices linked to this paper:

search_articles(query="PMID[PMID] AND (Retracted Publication[pt] OR Retraction of Publication[pt])")

Also check the `article_types` field from Step 1 — if it contains "Retracted Publication", the paper has been formally retracted.

**If retracted → Trust Level 1 (Compromised)**

2b. Errata and Corrections

Search for corrections:

search_articles(query="PMID[PMID] AND (Published Erratum[pt] OR Correction[pt])")

Errata can be minor (typo in a table) or major (recalculated results that change conclusions). Read the erratum to distinguish:

• **Minor**: Trust level unchanged
• **Major** (changes key results or conclusions): Trust Level 3 (Caution advised)

2c. Expression of Concern

search_articles(query="PMID[PMID] AND Expression of Concern[pt]")

An Expression of Concern from a journal editor indicates an active investigation. **Trust Level 2 (Reliability concerns)** until resolved.

Step 3: Check for Contradicting Publications

This is the most important and most nuanced step. Many problematic findings are never formally retracted — they are contradicted by subsequent independent work.

3a. Find Citing Articles

find_related_articles(pmids=["PMID"], link_type="pubmed_pubmed", max_results=50)

This returns computationally similar articles. From these, search for contradiction signals.

3b. Search for Contradiction Signals

Search PubMed for articles that cite the original AND contain contradiction language:

search_articles(query="\"[key claim from title]\" AND (\"fail to replicate\" OR \"unable to reproduce\" OR \"does not\" OR \"do not support\" OR \"contradicts\" OR \"challenges\" OR \"reanalysis\" OR \"re-analysis\" OR \"not reproducible\" OR \"could not confirm\")")

Also search with the first author's last name + key topic terms:

search_articles(query="[FirstAuthor] [KeyTopic] AND (Comment[pt] OR Letter[pt] OR \"fail\" OR \"does not\")")

3c. Search Academic Databases

Use Consensus to find contradicting evidence:

consensus_search(query="does [key claim from paper] replicate? contradiction evidence")

3d. Check bioRxiv for Preprints

Contradictions sometimes appear first as preprints: