/cascadeflow

name: cascadeflow
description: Use when building, extending, or debugging AI agents with cascadeflow (agent runtime intelligence layer) — installing `cascadeflow` (Python) or `@cascadeflow/core`/`@cascadeflow/langchain` (TypeScript); using `CascadeAgent`, `ModelConfig`, harness APIs (`cascadeflow.init`, `cascadeflow.run`, `@agent` from `cascadeflow.harness`, `simulate`), `withCascade`/`CascadeFlow`; picking drafter+verifier pairs; per-step budget/compliance/KPI enforcement; quality validation; complexity pre-routing; tool execution and multi-turn agent loops; presets; decision traces; or wiring cascadeflow into LangChain, OpenAI Agents, CrewAI, PydanticAI, Google ADK, n8n, or Vercel AI SDK. Also when a user mentions "cascade", "drafter/verifier", "runtime intelligence", "in-process harness", "cost-optimized agent", "agent loop with cost control", is in the lemony-ai/cascadeflow repo, or found a bug in cascadeflow/integrations needing an upstream fix/PR.

cascadeflow

What it is

**Agent runtime intelligence layer.** An in-process harness that sits *inside* the agent execution loop (not at the HTTP boundary) and makes per-step decisions on cost, latency, quality, budget, compliance, and energy. Sub-5ms overhead. Works alongside LangChain, OpenAI Agents SDK, CrewAI, PydanticAI, Google ADK, n8n, and Vercel AI SDK.

Two complementary pieces:

1. **Cascading** — try a cheap "drafter" model first, validate quality, escalate to a "verifier" model only when needed (40–85% cost savings). 2. **Runtime intelligence (harness)** — instrument the agent loop with budget caps, KPI weights, compliance gates, and a full per-step decision trace.

Python (`pip install cascadeflow`) and TypeScript (`@cascadeflow/core`). Docs: https://docs.cascadeflow.ai

Why "in the loop" matters (the core pitch)

cascadeflow is **not a proxy or a gateway**. It runs inside the agent's process and sees every model call, tool call, and sub-agent handoff as it happens — so it can act on running state (cost so far, tool calls used, compliance flag) at *each step*, not just per HTTP request.

| Dimension | External proxy | cascadeflow harness | |---|---|---| | Scope | HTTP request boundary | Inside the agent loop | | What it can see | One request at a time | Full run state (cost-so-far, step #, tool-calls used, budget remaining) | | Optimization axes | Cost only | Cost · latency · quality · budget · compliance · energy — simultaneously | | Latency overhead | 10–50 ms network RTT per call | <5 ms in-process per call | | 10-step agent loop | +400–600 ms avoidable | negligible | | Enforcement | Observe only | `allow` · `switch_model` · `deny_tool` · `stop` | | Auditability | Request logs | Per-step decision trace (one entry per LLM/tool/handoff decision) | | Business logic | None | Live KPI weights + targets injected at runtime |

This is what unlocks: stop-after-step-7 budget enforcement, deny-this-tool-mid-loop, switch-models-on-this-call, and a full audit trail of *why* every step did what it did. None of that is possible from outside the loop.

When to use this skill

• User is building an AI agent and wants cost/latency/quality control *inside* the loop
• Code imports `cascadeflow`, `@cascadeflow/core`, `@cascadeflow/langchain`, `@cascadeflow/vercel-ai`, or `@cascadeflow/n8n-nodes-cascadeflow`
• Mentions budgets, compliance (GDPR/HIPAA/PCI), KPI weights, tool-call routing, decision traces, drafter/verifier — *together with* a cascadeflow signal (import, repo path, or explicit cascadeflow mention). Don't fire on unrelated compliance/budget conversations in user code.
• Working inside `lemony-ai/cascadeflow` (examples, integrations, gateway server)
• A bug is discovered in cascadeflow itself or any of its integrations and needs to be fixed upstream

Pick the right entry point (30-second decision)

| Situation | Use | File/pattern | |---|---|---| | Existing OpenAI/Anthropic app, want instant observability | `cascadeflow.init(mode="observe")` | Auto-patches the SDKs. Zero code changes in the app. | | Existing app, no code changes at all, want gateway | `python -m cascadeflow.server` | Drop-in OpenAI/Anthropic-compatible proxy; point client at `http://127.0.0.1:<port>/v1` | | New agent, want the default "just works" cascade | `auto_agent()` or `get_cost_optimized_agent()` | Presets — fastest path; no model picking required | | New agent, custom drafter+verifier | `CascadeAgent(models=[drafter, verifier])` | Both languages | | Agent function with budget + policy metadata | `from cascadeflow.harness import agent` then `@agent(budget=..., compliance=..., kpi_weights=...)` | Attaches metadata; combine with `cascadeflow.run()` for enforcement. Note: import the decorator from `cascadeflow.harness` — `cascadeflow.agent` resolves to the module, not the decorator. | | Scoped run with budget and full trace | `with cascadeflow.run(budget=0.50, max_tool_calls=10) as session:` | Primary harness pattern | | Inside LangChain / OpenAI Agents / CrewAI / PydanticAI / Google ADK / Vercel AI / n8n | Use the integration package | Don't reinvent — the integrations preserve tool calling, streaming, callbacks |

Minimum viable cascade

**Python:**

from cascadeflow import CascadeAgent, ModelConfig

agent = CascadeAgent(models=[
    ModelConfig(name="gpt-4o-mini", provider="openai", cost=0.000375),  # drafter
    ModelConfig(name="gpt-4o",      provider="openai", cost=0.00625),   # verifier
])

result = await agent.run("What's the capital of France?")
print(result.content, result.model_used, result.total_cost, result.cost_saved)

**TypeScript:**

import { CascadeAgent } from '@cascadeflow/core';

const agent = new CascadeAgent({
  models: [
    { name: 'gpt-4o-mini', provider: 'openai', cost: 0.000375 },
    { name: 'gpt-4o',      provider: 'openai', cost: 0.00625  },
  ],
});

const r = await agent.run('What is TypeScript?');
console.log(r.modelUsed, r.totalCost, r.savingsPercentage);

**Even faster — preset