/tokf-filter

name: tokf-filter
description: This skill should be used when the user asks to "create a filter", "write a tokf filter", "add a filter for <tool>", "how do I filter output", or needs guidance on tokf filter step types, templates, pipes, or placement conventions.
version: 0.1.0

tokf Filter Authoring

You are an expert at writing tokf filter files. tokf is a config-driven CLI that compresses command output before it reaches an LLM context. Filters are TOML files that define how to process a command's output.

When the user asks you to create or modify a filter, follow this guide exactly. Produce valid, idiomatic TOML that matches the schema described below.

---

Section 1 — What a Filter File Is

A filter file is a TOML file that describes:

• Which command(s) it applies to (`command`)
• How to transform the raw output (steps, applied in a fixed order)
• What to emit on success vs. failure

Filters live in three places, searched in priority order:

1. `.tokf/filters/` — project-local (repo-level overrides) 2. `~/.config/tokf/filters/` — user-level overrides 3. Built-in library (embedded in the tokf binary)

First match wins. Use `tokf which "cargo test"` to see which filter would activate for a given command.

---

Section 2 — Processing Order

Steps execute in this fixed order — **do not rearrange them**:

1. **`match_output`** — whole-output substring checks; if matched, short-circuits the entire pipeline and emits immediately 2. **`[[replace]]`** — per-line regex transforms applied to every line, in array order 3. **`strip_ansi` / `trim_lines`** — per-line cleanup (ANSI stripping, whitespace trimming) 4. **`skip` / `keep`** — line-level filtering (drop or retain lines by regex) 5. **`dedup` / `dedup_window`** — collapse duplicate consecutive lines 6. **`lua_script`** — Luau escape hatch; runs after dedup, before JSON/section/parse 7. **`[json]`** — JSON extraction via `JSONPath`; when configured, replaces section/parse/chunk 8. **`[[section]]` OR `[parse]`** — structured extraction (these are mutually exclusive; section is a state machine, parse is a declarative grouper). Skipped when `[json]` is configured. 9. **`[[chunk]]`** — block-based structured extraction with per-block aggregation, grouping, and tree output (runs on raw output, alongside sections). Skipped when `[json]` is configured. 10. **Exit-code branch** — `[on_success]` or `[on_failure]` depending on exit code 11. **`[fallback]`** — if neither `on_success` nor `on_failure` produced output 12. **`strip_empty_lines` / `collapse_empty_lines`** — post-processing cleanup on the final output

Within `[on_success]` and `[on_failure]`, fields are processed as:

• `head` / `tail` → trim lines
• `skip` / `extract` → further filter
• `aggregate` → reduce collected sections
• `output` → final template render

---

Section 3 — Top-Level Fields Reference

| Field | Type | Default | Description | |---|---|---|---| | `command` | string or array of strings | required | Command pattern(s) to match. Supports `*` wildcard. | | `run` | string | (same as command) | Override the actual command executed. Use `{args}` to forward arguments. | | `match_output` | array of tables | `[]` | Whole-output checks. Short-circuit on first match. | | `[[replace]]` | array of tables | `[]` | Per-line regex replacements, in order. | | `skip` | array of strings (regex) | `[]` | Drop lines matching any regex. | | `keep` | array of strings (regex) | `[]` | Retain only lines matching any regex. (Inverse of skip.) | | `dedup` | bool | `false` | Collapse consecutive identical lines. | | `dedup_window` | integer | `0` (off) | Dedup within a sliding window of N lines. | | `strip_ansi` | bool | `false` | Strip ANSI escape sequences before skip/keep. | | `trim_lines` | bool | `false` | Trim leading/trailing whitespace from each line. | | `lua_script` | table | (absent) | Luau escape hatch. | | `[json]` | table | (absent) | JSON extraction via `JSONPath`. When configured, replaces `[[section]]`/`[parse]`/`[[chunk]]`. | | `[[section]]` | array of tables | `[]` | State-machine section collectors. | | `[[chunk]]` | array of tables | `[]` | Block-based structured extraction with per-block aggregation and grouping. | | `[parse]` | table | (absent) | Declarative structured parser (branch + group). | | `[on_success]` | table | (absent) | Output branch for exit code 0. | | `[on_failure]` | table | (absent) | Output branch for non-zero exit. | | `[output]` | table | (absent) | Top-level output template (used by `[parse]`). | | `[fallback]` | table | (absent) | Fallback when no branch matched. | | `strip_empty_lines` | bool | `false` | Remove all blank lines from the final output. | | `collapse_empty_lines` | bool | `false` | Collapse consecutive blank lines into one. | | `show_history_hint` | bool | `false` | Append a hint line after filtered output pointing to the full output in history. | | `[[variant]]` | array of tables | `[]` | Context-aware delegation to specialized child filters. |

---

Section 4 — Step Types

4.1 `match_output` — Whole-Output Short-Circuit

Check the entire raw output for a substring. If matched, emit a fixed string and stop — no further processing.

match_output = [
  { contains = "Everything up-to-date", output = "ok (up-to-date)" },
  { contains = "rejected", output = "✗ push rejected (try pulling first)" },
]

• `contains`: literal substring to search for (case-sensitive)
• `output`: string to emit if matched
• `{line_containing}` template variable: the first line that contains the substring

match_output = [
  { contains = "error", output = "Error on: {line_containing}" },
]

**When to use**: for well-known one-liner outcomes that make the rest of filtering irrelevant (e.g., "already up to date", "nothing to push", "authentication failed").

---

4.2 `[[replace]]` — Per-Line Regex Transforms

Applied to every line, in array order, before skip/keep. Use to reformat noisy lines.

[[replace]]