/distill-feedback

name: distill-feedback
description: Turn captured user-correction signals into durable rules (learn-from-corrections loop). Use when - /distill-feedback, "process feedback queue", "what corrections did I give you", "encode lessons from my corrections", session-feedback-capture queued sessions, "обнови правила по моим поправкам", "разбери очередь обратной связи". Reads ~/.claude/feedback/queue.jsonl, LLM-semantically detects durable corrections, proposes atomic rules, applies human-gated via delta-merge. Do NOT use to act on a single in-session correction (just apply the fix directly) or to hand-edit settings.json behaviors; this only mines the queued feedback backlog into durable rules.

distill-feedback — close the learn-from-corrections loop

The Stop hook `session-feedback-capture.py` queues finished sessions into `~/.claude/feedback/queue.jsonl`. This skill processes that queue: it finds the user turns that were **durable corrections** of the agent's work and turns them into rules — so the same correction never has to be given twice.

**Why semantic review:** standing preferences depend on context, not a trigger-word count. The formerly cited private `effectiveness-test/RESULTS.md` was absent when checked on 2026-09-06; its F1 claims are withdrawn from this skill until the dataset, labels, model, held-out split and raw predictions can be inspected. The rubric below is a review method, not a demonstrated accuracy guarantee. Do not substitute another paper's scores for our own.

**Research and authority:** [ACE](https://arxiv.org/abs/2510.04618) studies evolving context; [TRACE](https://arxiv.org/abs/2606.13174) studies compiling corrections into runtime checks. Neither proves this local extractor's accuracy. The approval boundary comes from our applicable user instructions and `autonomy-risk-tiers.md`, not an inferred paper mandate. New standing rules are proposals; an already authorized correction to an existing rule can be implemented within that exact authority. Codex memory changes also require the separate explicit user request and supported memory-update channel.

Procedure

1. Extract the queue (deterministic)

python ~/.claude/skills/distill-feedback/scripts/extract_feedback_queue.py --limit 8

Returns JSON: `{pending, sessions:[{session_id, cwd, ts, user_turns:[...]}]}`. `--limit` bounds the LLM pass (billing: distillation is opt-in, not every-session). If `pending` is 0, stop — nothing to do.

2. Detect durable corrections (LLM-semantic, prefer a fresh sub-agent)

For independence (Generator-Evaluator), spawn a sub-agent with the **rubric** below and the extracted `user_turns`. Ask it to return, per genuine correction: `{quote, durable_rule, applicability_condition, confidence, session_id}`. Pass only the turns — not your own reasoning.

**RUBRIC — a user turn is a DURABLE CORRECTION** if the user pushes back on / redirects the agent's behavior in a way that implies a STANDING preference or a mistake to avoid in future:

• explicit pushback / redirection ("no, do X instead", "wrong file again")
• reminder of a prior agreement ("we agreed you'd ask first", "мы же договаривались сначала бэкап")
• standing-preference marker ("from now on / always / never / by default / в следующий раз / впредь")
• frustration at a REPEATED mistake ("опять", "again", "you keep")
• polite redirection phrased as a question ("could you not overwrite latest.pth each time?")
• revert with a reason ("верни как было, твоя версия хуже")
• **praise THEN correction — judge the whole turn** ("great it runs, but always pin versions" = YES)

**NOT a durable correction:** new feature/task request · diagnostic question ("why did the build fail?", "почему-то падает") · factual/info statement even with "should be / by default / never" ("deploy should be done in 5 min", "по умолчанию там 8080") · agreement ("actually that makes sense, go ahead") · reassurance ("don't worry about the tests") · praise-only · off-topic chatter.

3. Dedup + draft atomic rules

For each detected correction: write it as ONE atomic rule with an applicability condition. **Dedup against existing rules/memory** (`grep` `~/.claude/rules/` and the project memory) — if it is already a rule, skip or propose an EDIT, not a new ADD. Cluster duplicates across sessions into one rule.

4. Resolve authority for the exact change

Show the user a compact table: each proposed rule + its applicability condition + source quote + target file + action (ADD new / EDIT existing / SUPERSEDE old / SPLIT). If that exact change is not already authorized, ask for approval and retain the proposal. Do not repeatedly request permission already given for the same in-scope correction. New always-on rules, `SUPERSEDE` and `DELETE` require the applicable explicit authority.

5. Apply (delta-merge, never rewrite)

On approval, apply each accepted delta with the ACE discipline from `memory-maintenance.md`: addressable ADD/EDIT only, dedup, preserve nuance (no full-file rewrite). Put it in the right home (`file-organization-cohesion.md`): a global rule → `~/.claude/rules/`, a project-specific lesson → that project's memory/CLAUDE.md. **If the rule is mechanically checkable** (file-name shape, forbidden command, tool-call form), note that it should graduate to a hook/validator (deterministic tier beats prose — `learn-from-corrections.md`).

6. Mark only reconciled sessions processed

First account for every selected queue item: inspected transcript, accepted/rejected corrections and their resulting artifact, or an explicit unresolved evidence gap. The extractor's `pending` is the selected window when `--limit` is used, not whole-queue completion. A missing/unreadable transcript is not processed; locate the canonical private chat archive by session id and inspect the recovered transcript before closing it. Only ids actually reconciled in this pass go into the following command:

python ~/.claude/skills/distill-feedback/scripts