unbrowse

Unbrowse

The Unbrowse client boundary is open and auditable. The local runtime, CLI bridge, SDK, drop-in adapters, and wallet/auth/signing layer are MIT and readable here, so you can verify what runs on your machine rather than trust a black box. The backend owns the route graph, ranking, settlement, and recursive contract compilation; the client sees only typed holes, approvals, pointer-only receipts, and wallet-sealed values. Company route IP stays behind typed contracts: end users ask for results, not raw internal API maps, auth material, HAR payloads, or PII. Inspect the live bridge contract at GET /v1/contract/surface. See docs/OPEN-SOURCE-NOTICE.md for the exact open/private split.

Unbrowse is the route layer / action layer for AI agents — a local Agent Skill, CLI, and TypeScript SDK that learns first-party website routes from real browsing, reuses them on later calls, and keeps a real browser only when the site still requires it. Credentials stay local; only sanitized route metadata is shared with the marketplace when you explicitly publish. MCP remains available as a compatibility surface.

The working claim is deliberately narrow: if a site already exposes a first-party route behind its UI, an agent should reuse that route instead of rediscovering it through a browser on every call.

The route graph is also a compensation surface. Routes are maintained assets, not anonymous scraped blobs: indexers can be paid when their routes are reused, site-owner splits are supported where claimed, and credentials stay local through pointer-only receipts and wallet-sealed values. When a call depends on an unavoidable paid upstream, Unbrowse keeps the settlement path explicit instead of hiding the cost in an agent loop. Details live in docs/HOW_UNBROWSE_PAYS.md.

The measured result in the first paper is a 3.6× mean speedup and 5.4× median speedup across 94 live domains when warmed cached routes replace browser automation, with sharply lower token use because the agent receives structured data instead of a page dump. See arXiv:2604.00694. For current release-coverage methodology (corpus shape, rubric, current numbers), see docs/benchmarks.md.

On adversarial, JavaScript-challenge-gated anti-bot content, a reproducible nine-post retrieval benchmark across three communities of a major social platform — ground-truthed against the platform's own data — recovers the real content on 9/9 posts where a naive HTTP client is blocked on every request (HTTP 403). The benchmark is re-runnable and reports the naive-vs-Unbrowse head-to-head directly.

On a live-harvested adversarial corpus — 24 sites mined each run from r/webscraping (the domains practitioners report fighting) plus curated vendor-gated, SPA, and GraphQL targets — the shipped binary's API-native resolve→execute path covers 12/24 (50%) on retrieval, with the misses concentrated on JavaScript-challenge and commercial anti-bot gates (Cloudflare, DataDome) that route through the browser-capture path rather than the thin API path. The credential-redaction / no-secret-on-the-wire security invariant holds across all 24 sites, including the blocked ones. The corpus is re-harvested and re-scored each run, anti-bot misses are recorded with their vendor class (never relabelled a pass), and every number is a gate that exits 0 only when the run was honest — see docs/benchmarks.md.

Security note: capture and execution stay local by default. Credentials stay on your machine. Learned API contracts are only shared after an explicit checkpoint (sync, close, or manual publish). Agents should connect through the installed Agent Skill or the SDK hole surface.

A Uniform Agent Interface

The current client boundary is a hole/contract: the model fills only the holes it can know, and the runtime chooses the cheapest capable layer. The formal bridge is machine-readable:

curl https://beta-api.unbrowse.ai/v1/contract/surface

The bridge exposes five client-fillable holes:

• intent
• wallet_proof
• approval
• local_capability_result
• typed_pointer

In the CLI and SDK this is one tool:

unbrowse "the top Hacker News stories with points"
unbrowse "the top Hacker News stories with points" --url "https://news.ycombinator.com"

import { createHole } from "unbrowse/sdk";

const hole = createHole();
const result = await hole.fill({
  intent: "get the top Hacker News stories with points",
  url: "https://news.ycombinator.com",
});

Internally the runtime may resolve a route, execute a captured endpoint, call a standard adapter, open a browser, reuse local cookies, inspect HAR, capture a new route, and index it. The agent-facing contract is the hole, not that internal ladder.

The explicit CLI surface is flat top-level commands (not build / act / eval prefixes):

Each op produces a pointer-only, wallet-signed receipt: it points at values (a URL, a value:ptr, a sha256: address) and carries a signature from your key — it never carries the secret value itself. Credential fills dereference locally; the secret never crosses the wire. We never see your secret values.

Receipts are Ed25519-signed today. Stronger authorization and provenance schemes are an active research direction; specifics will be detailed in a forthcoming whitepaper. The pointer-only invariant holds regardless. Full public surface — the hole contract, compatibility ops, the receipt shape, and the honest open/closed split — is in docs/agent-internet-layer.md.

The bare unbrowse "task" front door is the primary path. Legacy three-verb prefixes (build / act / eval / act) may still parse as aliases but print a deprecation notice — prefer the flat table above.

The contract platform

The hole/contract surface is one face of a uniform contract platform: every unit of work — resolving an intent, executing a route, sealing a value, settling a payment — is the same kind of object (a declared, signed, cached, accountable truth-claim), so one set of rules for identity, caching, and accountability applies everywhere. You declare a goal; the in-process runtime compiles it into a contract, resolves it against real evaluators, signs the verdict with a wallet-derived key, and appends it to a content-addressed ledger. Re-resolving unchanged work is free because the cache keys on a signature chain over pointers, not on copied output bytes.

Full design — declare surface, wallet-bound signing, append-only + on-chain ledger, pointer-keyed cache, single-source-of-truth binding, and what ships vs what is forward-looking — in docs/concepts/contract-platform.md.

Drop-in client adapters

Already using a search or browsing client? Swap one import. Unbrowse ships drop-in adapters that mirror the call shapes of exa-js, @tavily/core, and browser-use, all routed through a single streaming hole contract (resolve → execute → capture; a browser opens only as a fallback) that can be wallet-bound so each request is Ed25519-signed:

import Exa from "unbrowse/sdk/adapters/exa";        // was: import Exa from "exa-js"
const { results } = await new Exa(key).search("anthropic news", { numResults: 5 });

Full surface (exa / tavily / browser-use + the wallet-protected hole tool): docs/adapters.md.

Install — pick one

Option 1 — Agent Skill + CLI

Install the binary, then run setup. Setup installs the Unbrowse Agent Skill by default and does not write MCP host configs.

npm i -g unbrowse@latest
unbrowse setup

Skill-aware hosts read ~/(internal) and learn the current hole/contract surface. For legacy MCP hosts, run the stdio server manually:

unbrowse mcp

unbrowse mcp remains a manual stdio compatibility server for hosts that still need MCP.

Option 2 — TypeScript SDK

One SDK, one install. The current SDK surface is the hole: createHole().fill(...).

npm i unbrowse

import { createHole, mergedAuthHeaders } from "unbrowse/sdk";

// Web3-native: the wallet signature is the sole required credential.
// mergedAuthHeaders() reads the local wallet at ~/.unbrowse/wallet.json
// and emits the three auth capability headers per request. The backend
// verifies the sig and authenticates as `wallet:<pk>` BEFORE any bearer
// path — a wallet-only caller is a full principal, never key-gated.
const hole = createHole({ client: { walletSigner: mergedAuthHeaders } });
const data = await hole.fill({
  intent: "search Hacker News for AI agent papers",
  url: "https://news.ycombinator.com",
});

A local self-custody ed25519 wallet is auto-created on first run — no signup required. The same wallet signs x402 payment envelopes, so "who you are" and "who pays" are the same handle.

Optional web2 wrapper (deprecated). For account-bound flows (payouts accrual, dashboard sync, ToS surface tied to an email), layer a ubr_ api-key over the wallet:

const hole = createHole({
  client: { walletSigner: mergedAuthHeaders, apiKey: process.env.UNBROWSE_API_KEY },
});

Register at unbrowse.ai/login?cli=1 for an optional bound account key. The same install also provides the unbrowse CLI and legacy MCP server (npx unbrowse mcp) — see SKILL.md for the full surface.

Option 3 — Standalone CLI

If you just want the binary on your machine:

curl -fsSL https://unbrowse.ai/install.sh | sh

The installer detects your platform, downloads the matching release tarball, installs unbrowse into ~/.local/bin, then runs unbrowse setup.

For OpenClaw / agent-browser users, the plugin form is also still around — npx unbrowse-openclaw install --restart routes every page.goto() through Unbrowse — but it is no longer the primary install path.

Option 4 — Drop-in shim for an existing browser tool

If you already have a codebase on Playwright, Firecrawl, or Browserbase Stagehand, change one import line:

- import { chromium } from 'playwright';
+ import { chromium } from '@unbrowse/playwright-shim';

- import Firecrawl from '@mendable/firecrawl-js';
+ import Firecrawl from '@unbrowse/firecrawl-shim';

- import { Stagehand } from '@browserbasehq/stagehand';
+ import { Stagehand } from '@unbrowse/stagehand-shim';

Every goto / scrape / act / extract short-circuits through the Unbrowse marketplace cache first. Cache hit → free synthesized response. Miss → falls through to the original library (kept as an optional peer dep) so your existing API key still works. You pay the original vendor only when we miss.

Side-by-side on each: /compare/playwright, /compare/firecrawl, /compare/browserbase.

How payments work