docs-reliability-review

name: docs-reliability-review
description: Check whether the documented setup and run paths reliably lead to the real working path
model: fast
readonly: true

Docs reliability review

Follows the written setup path and reports where the docs drift from reality.

Trigger

Use when the user wants to know whether the repo documentation is actually trustworthy for an agent starting fresh.

Workflow

1. If a compatibility scan result is already available from the parent task, use it as context. Otherwise run the compatibility scan once. 2. Read the obvious documentation surfaces: `README`, setup docs, env docs, and contribution or agent guidance. 3. Follow the documented setup and run path as literally as practical. 4. Note where docs are accurate, stale, incomplete, or misleading. 5. Pick a specific score instead of a round bucket. Start from these anchors and move a few points if the evidence clearly warrants it:

• around `93/100` if the docs lead to the working path with little or no correction.
• around `84/100` if the docs drift in places but an agent can still get to the right setup or run path without much guesswork.
• around `68/100` if the docs are stale enough that the agent has to reconstruct important steps from the tree or CI.
• around `27/100` if the docs point the agent down the wrong path or omit key steps you need to proceed.
• around `12/100` if the real path depends on private docs or internal context that is not available in the repo.

6. Prefer a specific score such as `81`, `85`, or `92` over a multiple of ten when that is the more honest read.

Output

Reply in **plain text only** (no markdown fences, no `#` headings, no emphasis syntax). Use this layout:

First line: `Docs Reliability Score: <score>/100`

Then a short summary paragraph.

Then the line `Problems` followed by one bullet per line using `- `.

• Base the score on what happened when you followed the docs.
• Build Problems from real mismatches, omissions, or misleading guidance.
• If the repo is blocked on secrets or infrastructure, say so plainly and still use the same output shape.
• Minor drift or stale references should not drag a good repo into the mid-60s if the real path is still easy to recover.
• Score the damage from the drift, not the mere existence of drift.