/cao-agent-plugin-audit

name: cao-agent-plugin-audit
description: Audit an existing Agent Plugins 1.0.0 package - check its plugin.json manifest and mcp.json against CAO's vendored schemas, and mechanically check every Agent Skill it ships against the name pattern, the 64-character name bound, folder-equals-name, the 1024-character description limit and the 500-line body limit, four of which CAO's validator only warns about. Use when the user says "audit this Agent Plugin", "is this package conformant", "check these skills against the Agent Skills limits", "cao plugin validate says loadable, is it really fine", or "run gate 1 over skills/". Its validator script needs no CAO install and opens no network connection itself; its first run needs its three declared dependencies available. Not for authoring a new Agent Plugin (use cao-agent-plugin-creator). Not for CAO event plugins, the Python packages that react to cao-server lifecycle events (use cao-plugin). Not for auditing a client-specific agent plugin before migrating it (use migrate-agent-plugin).

Audit an Agent Plugin

Check an existing Agent Plugins 1.0.0 package against two things: the vendored manifest schemas, and the Agent Skills limits. The second half is the reason this skill exists. **CAO does not enforce four of the five skill rules below (the name pattern, the 64-character name bound, the 1024-character description limit and the 500-line body limit). `cao plugin validate` reports them only as warnings and loads the skill anyway.**

This skill audits a package that already follows the Agent Plugins 1.0.0 structure. It is not a pre-migration review of a client-specific agent plugin; that is `migrate-agent-plugin`'s job.

What CAO checks, and what it does not

CAO's skill loader (behind `cao skills add`, the skill catalog and `cao plugin add`) skips a skill for exactly three things:

1. frontmatter `name` is missing or blank; 2. frontmatter `description` is missing or blank (`src/cli_agent_orchestrator/models/skill.py`); 3. the folder name differs from the frontmatter `name` (`src/cli_agent_orchestrator/utils/skills.py`).

**It loads a skill that breaks the Agent Skills name pattern, the 64-character name bound, the 1024-character description limit or the 500-line body limit.**

`cao plugin validate` reports each of those four as a `WARNING` with the same code this skill uses (`skill.name_pattern`, `skill.name_length`, `skill.description_length`, `skill.body_length`). A warning leaves the package loadable and the exit status at 0, and the skill is still loaded and delivered. CAO releases before issue #1154 report nothing for them: the over-limit experiment for issue #1153 (task T3) observed `"loadable": true` with `"findings": []` for a 2000-character description and for a 600-line body. Reproduce the length case in a temporary directory:

W=$(mktemp -d); mkdir -p "$W/pkg/skills/long-desc"
printf '{"$schema":"https://agent-plugins.org/schemas/1.0.0/plugin.schema.json","name":"pkg"}' > "$W/pkg/plugin.json"
printf -- '---\nname: long-desc\ndescription: %s\n---\nbody\n' "$(printf 'x%.0s' $(seq 1 2000))" \
  > "$W/pkg/skills/long-desc/SKILL.md"
CAO_AGENT_PLUGINS_ENABLED=1 cao plugin validate --json "$W/pkg"   # loadable true, one skill.description_length warning, exit 0
rm -rf "$W"

So a package that passes `cao plugin validate` has not been shown to meet the Agent Skills limits: read its warnings. Do not tell anyone CAO rejects an over-limit skill. This skill's validator reports the four limits as `LIMIT` findings that make it exit 1, needs no CAO install, and also checks a bare `skills/` directory.

Prior art: a checklist, by hand

This is not the first audit aid for Agent Plugins. The `migrate-agent-plugin` skill (CAO carries it as a conformance fixture under `test/agent_plugins/fixtures/canonical-example/`) ships `references/validation-checklist.md`, which covers naming **by hand**: one checkbox says the manifest `name` "satisfies the v1 length and character rules", another that the frontmatter `name` "matches the directory name and Agent Skills naming rules". It is 30 human checkboxes. **It mentions neither the 1024-character description limit nor the 500-line body limit.** This skill's addition is that the checks are mechanical, and that it covers those two limits.

The five skill rules

Every skill is checked against all five. The body is the text after the closing `---` of the frontmatter, stripped; a whole-file line count over-counts. The name pattern for rule 1 is:

^[a-z0-9]+(-[a-z0-9]+)*$

| # | Rule | Limit | What `cao plugin validate` does | |---|---|---|---| | 1 | Name pattern | the pattern above | warns, still loads | | 2 | Name length | at most 64 characters | warns, still loads | | 3 | Folder name equals frontmatter `name` | exact | skips the skill | | 4 | Description length | at most 1024 characters | warns, still loads | | 5 | Body length | at most 500 lines | warns, still loads |

The full rule set, every finding code and its severity are in [references/rules.md](references/rules.md).

Two validators, reported separately

This skill uses two validators and keeps their findings apart: CAO's appear under their own `cao.` codes, never as the bundled validator's.

**1. The bundled validator, `scripts/validate_agent_plugin.py`.** It is a single Python file with a PEP 723 block declaring `jsonschema` and `pyyaml`, both of which it imports, plus `referencing`, the library `jsonschema` uses to resolve schema references, which it also imports. Run it with `uv run --script`. **It does not need CAO installed, and it opens no network connection itself**: schema references resolve only against the two bundled schemas and any other retrieval is refused. Its first run needs its three declared dependencies available, which means a uv cache that already holds them or a reachable package index. It validates the manifest against a bundled copy of the schema bytes under this skill's `schemas/1.0.0/`, a