/review-ai-writing

name: review-ai-writing
description: "Detect AI-generated writing patterns in developer text — docs, docstrings, commit messages, PR descriptions, and code comments. Use when reviewing any text artifact for authenticity and clarity, or when the user mentions ai writing, ai-generated or robotic writing, text that sounds like AI or ChatGPT, or writing quality. Builds on the docs-style core writing principles."
disable-model-invocation: true

Review AI Writing

Detect AI-generated writing patterns across developer text artifacts, parallelizing across artifact groups when the agent supports it.

Usage

Invoke the **review-ai-writing** skill with optional flags: `review-ai-writing [--all] [--category <name>] [path]`.

**Flags:**

• `--all` - Scan entire codebase (default: changed files from main)
• `--category <name>` - Only check specific category: `content|vocabulary|formatting|communication|filler|code_docs`
• Path: Target directory (default: current working directory)

Instructions

1. Parse Arguments

Extract flags from `$ARGUMENTS`:

• `--all` - Full codebase scan
• `--category <name>` - Filter to specific category
• Path - Target directory

2. Load Skills

Load the [review-verification-protocol](../../../beagle-core/skills/review-verification-protocol/SKILL.md) skill before reporting findings. The AI-writing pattern catalog lives in this file's Reference Material section and the `references/*.md` files — read the categories you intend to check.

3. Determine Scope

# Default: changed files from main
git diff --name-only $(git merge-base HEAD main)..HEAD

# If --all flag: scan all text artifacts
find . -type f \( -name "*.md" -o -name "*.py" -o -name "*.ts" -o -name "*.tsx" -o -name "*.js" -o -name "*.jsx" -o -name "*.go" -o -name "*.rs" -o -name "*.java" -o -name "*.rb" -o -name "*.swift" -o -name "*.kt" -o -name "*.ex" -o -name "*.exs" \) ! -path "*/node_modules/*" ! -path "*/.git/*" ! -path "*/vendor/*" ! -path "*/__pycache__/*" ! -path "*/dist/*" ! -path "*/build/*"

If no files found, exit with: "No files to scan. Check your branch has changes or use --all."

4. Check for Existing LLM Artifacts Review

# Check if llm-artifacts review exists to avoid double-flagging
if [ -f .beagle/llm-artifacts-review.json ]; then
  echo "Found existing llm-artifacts review — will skip overlapping findings"
fi

Parse existing findings from `.beagle/llm-artifacts-review.json` if present. When consolidating, skip any finding where both the file:line and pattern type match an existing llm-artifacts finding (specifically `verbose_comment` and `over_documentation` types).

5. Classify Files by Type

Partition files into three groups:

| Group | File Types | Patterns to Check | |-------|-----------|-------------------| | **Prose** | `*.md` | All 6 categories | | **Code Docs** | `*.py`, `*.ts`, `*.tsx`, `*.js`, `*.jsx`, `*.go`, `*.rs`, `*.java`, `*.rb`, `*.swift`, `*.kt`, `*.ex`, `*.exs` | vocabulary, communication, filler, code_docs | | **Git** | Commit messages, PR descriptions | content, vocabulary, communication, filler |

For Git artifacts, collect recent commits:

# Commits on current branch not in main
git log --format="%H %s" $(git merge-base HEAD main)..HEAD

6. Scan Each Artifact Group

There are three artifact groups below (Prose, Code Docs, Git). **If the agent supports subagents** and total items >= 4, dispatch one subagent per in-scope group in parallel (up to 3); **otherwise** run the same group instructions sequentially yourself — identical output either way. If `--category` is set, handle only the matching category. Every subagent (or sequential pass) reads this skill's Reference Material and the relevant `references/*.md` patterns before scanning.

Group 1: Prose

**Scope:** Markdown files only **Check:** All 6 pattern categories **Instructions:** 1. Read each markdown file 2. Scan for all pattern categories 3. Apply the false positive checks from this skill 4. Return findings in the structured format

Group 2: Code Docs

**Scope:** Source code files **Check:** vocabulary, communication, filler, code_docs categories **Instructions:** 1. Extract docstrings and comments from each file 2. Scan for applicable pattern categories 3. Skip code itself — only check text in comments and docstrings 4. Return findings in the structured format

Group 3: Git

**Scope:** Commit messages and PR descriptions **Check:** content, vocabulary, communication, filler categories **Instructions:** 1. Read commit messages from the branch 2. If on a PR branch, read the PR description via `gh pr view --json body` 3. Scan for applicable pattern categories 4. Use synthetic paths: `git:commit:<sha>` with line 0, `git:pr:<number>` with line 0 5. Return findings in the structured format

7. Consolidate Findings

Wait for all subagents to complete, then:

1. Merge all findings into a single list 2. Remove duplicates (same file:line and type) 3. Remove findings that overlap with `.beagle/llm-artifacts-review.json` 4. Assign unique IDs (1, 2, 3...) 5. Group by category for display

8. Write JSON Report

Create `.beagle` directory if it doesn't exist:

mkdir -p .beagle

Write findings to `.beagle/ai-writing-review.json`:

{
  "version": "1.0.0",
  "created_at": "2025-01-15T10:30:00Z",
  "git_head": "abc1234",
  "scope": "changed",
  "files_scanned": 12,
  "commits_scanned": 5,
  "findings": [
    {
      "id": 1,
      "category": "vocabulary",
      "type": "ai_vocabulary_high",
      "file": "README.md",
      "line": 15,
      "original_text": "This library leverages cutting-edge algorithms to facilitate seamless data processing.",
      "description": "High-signal AI vocabulary: leverage, cutting-edge, facilitate, seamless",
      "suggestion": "This library uses streaming algorithms for fast data processing.",
      "risk": "Low",
      "fix_safety": "Safe",
      "fix_action": "rewrite"
    },
    {