/skill-upper

name: skill-upper
description: "Capture and review Agent Skill observations, and create, run, diagnose, or iteratively improve Skill evaluations (evals) with the skill-up CLI. Use when the user asks to record explicitly attributed Skill usage or feedback; review observations or user-supplied traces/transcripts; turn an approved observation into a regression case; evaluate, test, regress, verify, fix, improve, iterate, or evolve a Skill; add or strengthen eval cases; write eval.yaml/case.yaml; run skill-up run/validate/list-cases/report/import/init; or migrate from Anthropic evals.json. Plugin observation capture/review require either the Codex skill-up plugin or an observer-enabled DSH skill-up plugin. User-supplied external evidence can be reviewed without a plugin; evaluation remains multi-engine."

use-skill-up-cli

Help the user evaluate and evolve Agent Skills through the `skill-up` CLI.

Manual: <https://alibaba.github.io/skill-up/>

Language Policy

**Default to English when responding to the user. If the user writes in Chinese (or any other language), switch to that language and stay consistent with the user's input throughout the session.**

Detection rules (highest priority first):

1. The user explicitly specifies a language in the current message (e.g. "answer in English" or "reply in Chinese") → follow the user's instruction. 2. The natural language used in the user's current message → match it. 3. None of the above → use English (default).

Regardless of the response language, technical identifiers in this SKILL — CLI commands, `eval.yaml` / `case.yaml` field names, report field names, etc. — MUST stay in their original English form. Do not translate them.

Language Rules for Generated Artifacts

Use the user's current language for replies, newly authored explanatory prose, and YAML comments. Preserve the language and semantics of test inputs, expected output literals, identifiers, commands, paths, and existing regression cases. Do not translate a deterministic assertion merely because the conversation language changed. When testing localized behavior, keep the original business text even in an English conversation.

Treat `assets/*.tmpl` as structural references: adapt placeholder prose and comments to the requested language. Review only files created or changed for this task; never rewrite unrelated cases to enforce a language preference.

Use short field-leading comments for generated YAML. Explain actual configured fields: `schema_version`, `environment.type`, `engine.name`, `cases.files`, and `report.formats` in eval.yaml; `id`, `title`, `input.prompt`, and `judge.type` in case YAML. For nested fields, place the comment inside the parent mapping immediately before the child field, rather than only before the parent. For example, comment both `prompt` and `type` at their own indentation:

input:
  # User request sent to the agent.
  prompt: Review this code.
# Cheap prerequisites checked before the judge.
expect:
  must_contain: ["suggestion"]
judge:
  # Deterministic grading strategy for this case.
  type: rule_based

Comment `expect` and a top-level `judge` when present; do not add an optional field just to satisfy a comment check. Keep field names and enum values unchanged. `skill-up import` does not preserve template comments.

What is skill-up

`skill-up` is an evaluation CLI for Agent Skill authors. It installs the Skill into a real Agent Engine (Claude Code, Codex, qodercli, etc.), spins up an execution environment for each case, runs the prompt, then grades the result via declared rules / LLM judges / custom scripts, and finally produces a report.

Typical layout:

my-skill/
  SKILL.md
  evals/
    eval.yaml
    cases/
      <case-id>.yaml
    fixtures/

When to trigger

Use this skill in any of the following situations:

• The user asks to "run / evaluate / verify / test this skill".
• The user asks to "fix / improve / iterate / evolve this skill" from eval failures.
• The user wants to "add evals, test cases, or regression cases to a skill".
• The user asks to record explicitly attributed Skill usage, evidence, or feedback.
• The user wants to review captured Skill observations or turn approved feedback into a regression case.
• The user wants to edit `eval.yaml` / `case.yaml`, or asks you to choose an appropriate `judge` type.
• The user mentions `skill-up run/validate/list-cases/report/import/init`.
• The user wants to migrate from Anthropic `evals.json` to skill-up.
• The current working directory contains `evals/eval.yaml` or `evals/evals.json` and the user wants to run it.

Choose the requested workflow

• **Diagnose or plan:** read the target Skill and supplied evidence, identify

concrete problems, and propose changes or cases. Do not install tools, create eval files, change configuration, or run evaluations unless requested.

• **Create or edit cases:** use Steps 1–4 to create or update the suite and

validate its configuration. Stop before credentials and execution unless the user also requests a run. Report the actual config/case paths and validation status in the final reply.

• **Evaluate:** follow Steps 0–7 for the requested suite; report failures without

automatically editing the Skill.

• **Improve:** follow the evaluation workflow and Step 8 within the requested

scope. Preserve valid existing cases and assertions.

Evaluation workflow

Observation capture mode (optional)

Use this mode when the user asks to record the current Skill interaction or feedback and observation tools are available. It requires either a current Codex release with the `codex-skill-up` plugin and lifecycle-hook support, or DSH with the skill-up plugin's observer explicitly enabled. Do not retrofit observation capture onto the pinned Codex 0.80.0 evaluation adapter.

If the user asks how to install either host plugin, or the required observation tools are missing and the user asks to enable them, read `referenc