Skip to content
Content
Agent

tuning

**Navigation**: [Agents package rules](../AGENTS.md) → **Evaluation and Optimization**

From plugin
nodetool
63614 skills14 agents

Claude Code

$ npx -y skills add nodetool-ai/nodetool --agent claude-code

Agent definition

tuning.md

Evaluation and Optimization

**Navigation**: [Agents package rules](../AGENTS.md) → **Evaluation and Optimization**

Paths are relative to `packages/agents/` unless they name another package.

Cost Tracking

`CostCalculator` in `@nodetool-ai/runtime` tracks per-call costs based on provider pricing:

provider.trackUsage(model, { inputTokens: 100, outputTokens: 50 });
console.log(provider.getTotalCost()); // USD

Costs are logged via `logProviderCall()` and included in `llm_call` messages.

Model Selection

One run, one model. `create_plan` and `execute_plan` both use the calling session's provider and model (`subAgentRuntime`), so planning and step execution cannot drift apart or bill against a model the user did not pick. The per-phase model idea survives only as a possible argument on those two specs; nothing selects a second model.

Tool Result Truncation and Chat Compaction

A single result is cut, and a whole thread is summarized. Two different limits, in two different places.

  • **One tool result** is capped at 25 000 characters (`MAX_TOOL_RESULT_CHARS`

in `constants.ts`) before it enters history, with a notice telling the model the output was cut and how to ask for a smaller slice. A broad grep or a large file read otherwise returns tens of megabytes and the next request fails outright.

  • **A step transcript is never summarized.** Step executors hand the

tool-calling loop to `provider.generateLoop`, so each provider manages its own window — the Claude Agent SDK compacts internally, a stateless provider is sent everything. What bounds a step instead is the cumulative iteration cap and the result cut above.

  • **A chat thread is compacted**, in `packages/websocket`

(`session/chat-compaction.ts`, driven from `chat-turn.ts`). One summarizer call replaces everything before the last `NODETOOL_CHAT_COMPACTION_KEEP_TURNS` user turns, and the summary is persisted as a `role: "user"` row marked `execution_event_type: "compaction"`. History assembly starts from the newest such row, so only the provider's view is shortened — the full thread stays in the database for the UI and `nodetool.threads.*`. The cut lands on a user message, which is what keeps a tool call attached to its result.

  • **Two triggers**: the prompt reaching the threshold before the loop, and the provider itself

reporting that the prompt did not fit, which compacts and retries the turn once. A provider holding the transcript upstream skips the first, since shortening what NodeTool sends does not shorten what that provider already has.

  • **The threshold is 90% of the model's input window** unless

`NODETOOL_CHAT_COMPACTION_TOKENS` sets it. `resolveContextWindow` in `packages/runtime/src/providers/context-window.ts` asks the provider's Models API first (`BaseProvider.getContextWindow`), then models.dev by provider route and model, then falls back to 128 000 tokens with a warning. The chat turn caches the window per connection and `provider/model`. The prompt size is the larger of the local estimate and the provider-reported size of the thread's last call (`BaseProvider.lastCallTokens`).

  • **A failed summarizer leaves the thread uncompacted** and the turn runs

against the full history, with a `log_update` saying so. The alternative to an imperfect summary is a turn that cannot run at all.

  • **The user sees the cut.** The web renders the compaction row as a collapsed

"Earlier conversation summarized" card (`web/src/components/chat/message/CompactionCard.tsx`), reached from `MessageView` before the plain user-message path — without that branch the summary reads as something the user typed.

Plan Validation

Plans are validated before execution:

  • Step/task IDs must be unique across the entire plan
  • Dependencies must reference valid IDs
  • No circular dependencies (DAG validation via DFS)
  • On failure, the error is fed back to the model as the tool result, and it

calls `add_task` again

Output Schema Validation

Steps can enforce structured output via JSON schema:

  • `additionalProperties: false` enforced automatically
  • Schema'd steps finalize ONLY through the finishing tool — `finish_step` in tool mode, `finish()` in a CodeAct action. There is no JSON-from-text extraction path, so a step that never calls it fails and emits an explicit error result.
  • Unstructured steps (no schema) finalize when the model emits a no-tool-call assistant message; that text becomes the result.
  • Two things make that contract visible to a model that believes `return graph` finished the step, because the observation for the losing move used to be indistinguishable from success. A CodeAct observation for a schema'd step that returned a value without finishing carries `finished: false` and a `note` — which says so explicitly when the returned value already matched the schema. And a schema'd step whose turn ended in prose is re-prompted with the contract, at most `MAX_FINISH_NUDGES` (2) times, before it fails.
  • The failure message names the terminal state it actually hit: the provider's error, the run budget's own reason when a cost cap or deadline stopped the step, `exceeded N iterations` only when the loop really used its budget, else "ended after N action(s) / model turn(s) without calling finish", quoting the model's last message. A budget stop reported as "without calling finish" blamed the model for something it never got to do.
  • `maxIterations` bounds the **step**, not each finish-nudge round. Each round is given what is left of the allowance and a round with nothing left is not run, so `MAX_FINISH_NUDGES` no longer multiplies the ceiling — a step configured for 4 model turns used to be able to make 12.

Skills System

Skills come in two tiers.

A **user skill** is a row in the `skills` table — `name`, `description`, and markdown `content` — that someone wrote and can rewrite.

A **system skill** ships with the build: a `SKILL.md` under `pa

Read more
@nodetool-ai
Ships withnodetool

Open-source AI film studio · agent + node workflows Direct AI films, shot by shot. Start with one sentence. The agent writes the story, draws a still for every shot, and cuts the clips on a timeline.

6361 skill14 agents
Get the whole plugin

Stars on GitHub

Momentum

stars in total
636
Star history of nodetool-ai/nodetool on GitHub, a chart by star-history.com

Other agents on nodetool.

MUTATION_TESTING

MUTATION_TESTING

Agent

The agent package's pure, deterministic core (the chat permission gate, the control-tool plumbing, JSON extraction, and the…

cli

cli

Agent

**Navigation**: [Agents package rules](../AGENTS.md) → **Running Agents from CLI**

More agents from other plugins.

prompt-manager

prompt-manager

Agent

Agent for managing AI prompts on prompts.chat - search, save, improve, and organize your prompt library.

@gh-f@gh-fInstall Agent
humanize-diagnostician

humanize-diagnostician

Agent

정밀(strict) 모드 1단계 진단 에이전트. 글 전체를 한 번에 보고 "가장 지배적인 AI 티 패턴 3~6개"를 taxonomy ID와 함께 진단한다. 불안정한 span 열거(0↔18개로 요동) 대신 "무엇이 이 글을…

@epoko77-ai@epoko77-aiInstall Agent
blog-cluster-curator
Auto-invoked

blog-cluster-curator

Agent

Topic Clusters curator for the Claude Blog Brain. Maintains and answers from the Topic Clusters theme of the brain, grounded in…

@agricidaniel@agricidanielInstall Agent

Questions about tuning.

Which coding agents can use tuning?
tuning installs in Claude Code and Cursor: each loads subagents from .claude/agents, so copy the file there. Flowy's auto-invocation runs in Claude Code only.