/provenance
Analyze saved trajectories and recall audit events offline to record whether recalled guidelines influenced completed sessions.
$ npx -y skills add AgentToolkit/altk-evolve --skill provenance --agent claude-codeHow it fires
How this skill gets triggered: by you, by Claude, or both.
- Fires itselfAuto-invocation. Claude auto-loads it when your prompt matches the work.
- You can call itInvoke it directly when you want it.
- Slash command
/provenance
Context preview
The summary Claude sees to decide when to auto-load this skill.
Analyze saved trajectories and recall audit events offline to record whether recalled guidelines influenced completed sessions.
SKILL.md
provenance.SKILL.mdname: provenance
description: Analyze saved trajectories and recall audit events offline to record whether recalled guidelines influenced completed sessions.
Provenance Analyzer
Overview
This skill runs after one or more sessions have completed. It reads `recall` events from `.evolve/audit.log`, locates each session's trajectory, and records post-hoc `influence` events for the recalled guidelines.
The mechanical work — reading recall rows, skipping already-assessed pairs, resolving entity files, and locating trajectories — is done deterministically by `provenance.py candidates`. Your job is the judgment: read each candidate and decide whether the recalled guideline was `followed`, `contradicted`, or `not_applicable`, then persist that verdict.
Use this skill when you want to compute usage provenance without coupling the work to the live learn step.
Workflow
Step 1: Get candidates
Run the candidate builder. It emits one JSON object per line (JSONL), one per unresolved `(session_id, entity)` recall pair:
python3 ${CLAUDE_PLUGIN_ROOT}/skills/evolve-lite/provenance/scripts/provenance.py candidatesEach candidate looks like:
{
"session_id": "<session-id>",
"entity_id": "<type>/<name>",
"entity_excerpt": "<frontmatter + content of the entity file>",
"trajectory_path": "/path/to/transcript.jsonl",
"trajectory_excerpt": "<head of the trajectory transcript>",
"missing": ["trajectory"]
}Notes:
- `entity_id` is the path relative to `.evolve/entities/` without the `.md`
suffix, e.g. `feedback/foo`, `guideline/bar`, or `subscribed/alice/guideline/baz`.
- Pairs that already have an `influence` row are skipped for you — the builder
reuses the same dedup rule used when influence rows are written. You will never be handed a duplicate.
- The trajectory locator checks `.evolve/trajectories/` first, then falls back
to the native Claude transcript at `~/.claude/projects/<slug>/<session-id>.jsonl`. This means provenance works even when no `.evolve/trajectories/` file was written.
- If an entity file or trajectory cannot be found, the candidate is still
emitted with a `missing: [...]` field so the gap is visible. When the trajectory is missing you usually cannot judge the pair — skip it (do not guess), unless the entity content alone makes `not_applicable` certain.
Step 2: Judge each candidate
For each candidate, read `entity_excerpt` (and open `trajectory_path` for the full transcript if the excerpt is not enough). Compare the recalled guideline against the agent's actual actions in the trajectory and pick exactly one verdict:
- `followed` — the agent's actual actions are consistent with the guideline.
- `contradicted` — the guideline applied, but the agent did the opposite or
repeated the avoidable dead end.
- `not_applicable` — the guideline was recalled but did not apply to this
session.
Keep `evidence` to one short sentence citing a concrete action, tool call, or absence in the trajectory. This judgment is yours — there is no heuristic fallback.
Step 3: Record verdicts
Persist each verdict. Either pipe one verdict per call to `provenance.py record`:
echo '{
"session_id": "<session-id>",
"entity": "<type>/<name>",
"verdict": "followed",
"evidence": "Agent used the saved parser before trying shell fallbacks."
}' | python3 ${CLAUDE_PLUGIN_ROOT}/skills/evolve-lite/provenance/scripts/provenance.py record…or, to batch many assessments for one session in a single call, pipe to the underlying writer directly:
echo '{
"session_id": "<session-id>",
"assessments": [
{"entity": "feedback/foo", "verdict": "followed", "evidence": "Agent followed it."},
{"entity": "guideline/bar", "verdict": "not_applicable", "evidence": "Did not apply."}
]
}' | python3 ${CLAUDE_PLUGIN_ROOT}/skills/evolve-lite/provenance/scripts/log_influence.pyBoth paths write the identical `influence` audit row and skip duplicates. The `entity` value must match the candidate's `entity_id` exactly, including any `subscribed/<source>/` prefix.
It is valid to record nothing when recall events exist but no recalled guideline can be assessed (e.g. every candidate is missing its trajectory).
Read more
name: provenance description: Analyze saved trajectories and recall audit events offline to record whether recalled guidelines influenced completed sessions.
Provenance Analyzer
Overview
This skill runs after one or more sessions have completed. It reads `recall` events from `.evolve/audit.log`, locates each session's trajectory, and records post-hoc `influence` events for the recalled guidelines.
The mechanical work — reading recall rows, skipping already-assessed pairs, resolving entity files, and locating trajectories — is done deterministically by `provenance.py candidates`. Your job is the judgment: read each candidate and decide whether the recalled guideline was `followed`, `contradicted`, or `not_applicable`, then persist that verdict.
Use this skill when you want to compute usage provenance without coupling the work to the live learn step.
Workflow
Step 1: Get candidates
Run the candidate builder. It emits one JSON object per line (JSONL), one per unresolved `(session_id, entity)` recall pair:
python3 ${CLAUDE_PLUGIN_ROOT}/skills/evolve-lite/provenance/scripts/provenance.py candidatesEach candidate looks like:
{
"session_id": "<session-id>",
"entity_id": "<type>/<name>",
"entity_excerpt": "<frontmatter + content of the entity file>",
"trajectory_path": "/path/to/transcript.jsonl",
"trajectory_excerpt": "<head of the trajectory transcript>",
"missing": ["trajectory"]
}Notes:
- `entity_id` is the path relative to `.evolve/entities/` without the `.md`
suffix, e.g. `feedback/foo`, `guideline/bar`, or `subscribed/alice/guideline/baz`.
- Pairs that already have an `influence` row are skipped for you — the builder
reuses the same dedup rule used when influence rows are written. You will never be handed a duplicate.
- The trajectory locator checks `.evolve/trajectories/` first, then falls back
to the native Claude transcript at `~/.claude/projects/<slug>/<session-id>.jsonl`. This means provenance works even when no `.evolve/trajectories/` file was written.
- If an entity file or trajectory cannot be found, the candidate is still
emitted with a `missing: [...]` field so the gap is visible. When the trajectory is missing you usually cannot judge the pair — skip it (do not guess), unless the entity content alone makes `not_applicable` certain.
Step 2: Judge each candidate
For each candidate, read `entity_excerpt` (and open `trajectory_path` for the full transcript if the excerpt is not enough). Compare the recalled guideline against the agent's actual actions in the trajectory and pick exactly one verdict:
- `followed` — the agent's actual actions are consistent with the guideline.
- `contradicted` — the guideline applied, but the agent did the opposite or
repeated the avoidable dead end.
- `not_applicable` — the guideline was recalled but did not apply to this
session.
Keep `evidence` to one short sentence citing a concrete action, tool call, or absence in the trajectory. This judgment is yours — there is no heuristic fallback.
Step 3: Record verdicts
Persist each verdict. Either pipe one verdict per call to `provenance.py record`:
echo '{
"session_id": "<session-id>",
"entity": "<type>/<name>",
"verdict": "followed",
"evidence": "Agent used the saved parser before trying shell fallbacks."
}' | python3 ${CLAUDE_PLUGIN_ROOT}/skills/evolve-lite/provenance/scripts/provenance.py record…or, to batch many assessments for one session in a single call, pipe to the underlying writer directly:
echo '{
"session_id": "<session-id>",
"assessments": [
{"entity": "feedback/foo", "verdict": "followed", "evidence": "Agent followed it."},
{"entity": "guideline/bar", "verdict": "not_applicable", "evidence": "Did not apply."}
]
}' | python3 ${CLAUDE_PLUGIN_ROOT}/skills/evolve-lite/provenance/scripts/log_influence.pyBoth paths write the identical `influence` audit row and skip duplicates. The `entity` value must match the candidate's `entity_id` exactly, including any `subscribed/<source>/` prefix.
It is valid to record nothing when recall events exist but no recalled guideline can be assessed (e.g. every candidate is missing its trajectory).
Blog posts: IBM announcement | Hugging Face blog Coding agents repeat the same mistakes because they start fresh every session. Evolve gives agents memory — they learn from what worked and what didn't, so each session is better than the last.
Repo: AgentToolkit/altk-evolve
Other skills on altk-evolve.
- /agent-wiki-consolidate-guidelines
Read all atomic guidelines in wiki-twobatch/guidelines/ and propose themed clusters that group near-duplicates. Writes cluster pages and updates _config.yaml; originals are preserved with a `superseded_by:` backref.
Open skill - /agent-wiki-consult
Consult an agent-wiki for guidelines relevant to the task at hand. The wiki itself documents how to retrieve from it (AGENTS.md). Use this skill once you know what task or sub-task you're about to do — not at session start.
Open skill - /agent-wiki-extract-guidelines
Read a normalized Claude Code trajectory JSON and extract reusable guidelines into wiki-twobatch/guidelines/. Use when mining saved trajectories for reusable lessons.
Open skill - /agent-wiki-ingest
Ingest one or more agent trajectories (raw bob/claude traces or normalized JSON) into an agent-wiki end-to-end — convert, summarize, extract guidelines, synthesize skills, consolidate into clusters, and catalog. Use when you have a batch of traces to turn into a wiki in one pass.
Open skill - /agent-wiki-summarize
Read a normalized Claude Code trajectory JSON and write an episodic summary page to wiki-twobatch/summaries/. Use when summarizing one or more saved trajectories into the agent wiki.
Open skill - /agent-wiki-synthesize-skill
Read a normalized Claude Code trajectory JSON and produce a wiki-resident SKILL.md page that future agents can invoke. Use when a trajectory captured a non-trivial successful workflow worth promoting from a free-text guideline to an executable, callable artifact.
Open skill