/context-engineering

name: context-engineering
description: Use when managing what an LLM sees. Covers context-window budgeting, retrieval and compaction, memory across turns, tool-result pruning, and the failure modes that come from too much context rather than too little. Trigger on "model gets worse as the conversation gets longer", "context window keeps filling up". Do not use for state that persists across sessions — see the agent-memory skill; for retrieval over a document corpus, see the rag skill.
metadata:
  category: ai
  version: 1.0.0
  tags: [context, llm, memory, compaction, retrieval]

Context Engineering

Purpose

Decide what an LLM sees, and what it does not. Context is a finite budget with a nonlinear cost curve: more context is not more capability, and past a point it is actively less.

When to Use

• Building an agent or assistant that runs over many turns.
• A model that performs worse as the conversation gets longer.
• Managing tool results, retrieved documents, or long files.
• Reducing token cost in a long-running system.

Capabilities

• Context budgeting and allocation.
• Retrieval: putting the right thing in, not everything.
• Compaction and summarization of history.
• Memory: what persists across sessions, and in what form.
• Tool-result pruning and progressive disclosure.

Inputs

• The task, and what information it genuinely requires.
• The context window and its cost.
• The failure symptom, if the system is degrading over long runs.

Outputs

• A context budget with an allocation per component.
• A compaction strategy that preserves what matters.
• Measurably better performance on long tasks.

Workflow

1. **Budget the window explicitly** — System prompt, tools, retrieved context, history, and the response all compete for the same space. Decide the allocation rather than letting history consume everything. 2. **Retrieve narrowly** — Ten highly relevant chunks outperform a hundred marginally relevant ones. Irrelevant context does not sit inertly; it distracts. 3. **Compact, do not truncate** — Dropping the oldest turns loses the decisions that explain the current state. Summarize the history into the facts and decisions that are still live. 4. **Prune tool results** — A tool returning 50 KB of JSON when the agent needs three fields is spending the budget on noise. Filter at the tool boundary. 5. **Disclose progressively** — Provide a file listing, not the files. Let the model request what it actually needs. This is how a large codebase fits in a small window. 6. **Externalize memory** — Long-lived state belongs in a file or a store the model can read and write, not in a conversation history that grows without bound.

Best Practices

• Long context degrades attention. A model given 200,000 tokens of context often performs worse than the same model given the 5,000 tokens that mattered. Adding context is not free.
• Information in the middle of a long context is attended to least. Put the critical instruction at the start or the end, not buried in the middle.
• Compaction is lossy by design. Preserve the decisions, the constraints, and the unresolved questions; discard the exploration that led to them.
• A tool that returns raw API output is a context leak. Wrap it, select the fields, and return a summary with a way to fetch the detail.
• Cache the stable prefix. A system prompt and tool definitions that do not change should be cached rather than re-sent and re-processed each turn.
• Write important state to a file. A note the model writes and re-reads survives compaction; a fact buried in turn 12 does not.

Examples

**Explicit context budget:**

Window: 200,000 tokens. Allocation:

  System prompt + skill      4,000    stable, cached
  Tool definitions           3,000    stable, cached
  Working memory file        2,000    the agent's own notes, re-read each turn
  Retrieved context         20,000    top-k, re-retrieved per turn, not accumulated
  Conversation history      40,000    compacted when it exceeds this
  Tool results              30,000    pruned; only the most recent kept in full
  Reserve for response      16,000
                          --------
                           115,000    leaving deliberate headroom

Rules:
  - History exceeding 40k triggers compaction, not truncation.
  - Retrieved context is replaced each turn, never appended — otherwise it
    grows monotonically and crowds out everything else.
  - Tool results older than 3 turns are replaced by a one-line summary.

**Compaction that preserves what matters:**

COMPACTION_PROMPT = """\
Summarize the conversation so far into a working state document. This summary
replaces the full history, so anything you omit is lost permanently.

Preserve:
- The user's goal and any constraints they stated.
- Decisions made, and the reason for each.
- Facts discovered (file paths, API shapes, error messages, values).
- What has been tried and failed, so it is not retried.
- Open questions and the current next step.

Discard:
- Exploration that led nowhere.
- Full file contents (keep the path and what was learned from it).
- Tool call mechanics and intermediate output.

Write it as a factual state document, not a narrative."""

**Progressive disclosure instead of loading everything:**

# Wrong: 400 files, 2M tokens, does not fit and would not help if it did.
context = "\n".join(read(f) for f in repo.all_files())

# Right: give the model a map, and a tool to fetch what it decides it needs.
context = repo.tree(max_depth=3)          # ~2,000 tokens
tools = [read_file, grep, list_directory] # the model pulls what it needs

Notes

• The "lost in the middle" effect is well-documented: retrieval accuracy for a fact placed in the middle of a long context is substantially lower than for the same fact at either end. Placement is not neutral.
• Prompt caching makes a large stable prefix cheap after the first call. It changes the economics of a long system