/tune-eval
Evaluate a fine-tuned, distilled, or continued-pretrained model with held-out test discipline — the honest scoreboard at the end of the tunelab pipeline. Pre-registers the acceptance bar and metric set BEFORE results exist, runs the untouched test split through base and tuned
$ npx -y skills add rchaz/tunelab --skill tune-eval --agent claude-codeHow it fires
How this skill gets triggered: by you, by Claude, or both.
- Fires itselfAuto-invocation. Claude auto-loads it when your prompt matches the work.
- You can call itInvoke it directly when you want it.
- Slash command
/tune-eval
Context preview
The summary Claude sees to decide when to auto-load this skill.
Evaluate a fine-tuned, distilled, or continued-pretrained model with held-out test discipline — the honest scoreboard at the end of the tunelab pipeline. Pre-registers the acceptance bar and metric set BEFORE results exist, runs the untouched test split through base and tuned
tunelab moves repetitive LLM calls — tool calling, classification, extraction — onto small local models.
Other skills on tunelab.
- /tune-data
Build a training dataset for fine-tuning, distillation, or continued pretraining. Use when the user wants to turn logs/CSV/JSONL into fine-tuning data, label data with an LLM, distill a teacher model's outputs, generate synthetic training examples from nothing, chunk raw domain
Open skill - /tune-decide
The tunelab front door — decides whether a task needs fine-tuning at all, by running EXPERIMENTS on the user's data, not just interviewing. Use whenever the user wants to fine-tune, distill, or train a small/local model, cut their LLM API bill, replace frontier calls with
Open skill - /tune-loop
The tunelab capstone — drives a self-improving AI system. Use when the user wants a deployed model/cascade/workflow to keep getting better from feedback, run champion/challenger experiments to discover the best architecture, set up a data flywheel with retrain triggers, or
Open skill - /tune-train
Drive a local MLX-LM training run on Apple Silicon (LoRA/QLoRA, full fine-tuning, CPT) after tune-decide has validated a Level 2-3 plan. Use to pick a base model and hyperparameters, launch/monitor/resume a detached mlx_lm.lora run, diagnose loss curves, run continued
Open skill