clawdcursor

Safe desktop control for any AI agent. Compiles the screen into a UI map and acts on elements by stable id (screenshot/vision only as a last resort), verifies its own actions, and gates everything through one safety checkpoint. Local · cross-OS · any model.

Quickstart · What's new · Why it's different · The engine · How it works · Tools · Platforms · Changelog

What's new

1.5.24 — The AI never touches your pill, and failures stop looking like results.

• While a pause waits for you, nothing reads, clicks or types. The pill and pause prompt are never a target, even from another clawdcursor process.
• On macOS, a screenshot without Screen Recording permission is an error, not a wallpaper image, and open popovers are readable.
• The browser clawdcursor drives comes to the front when it navigates.
• A failed OCR read reports its error instead of "No text detected", and OCR confidence is only shown when the engine really has one.

1.5.23 — Windows: finding an element by name no longer matches stray letters. A lone "C" on another page could match "User code 0", and "Don't Save" could match Save. A partial match now has to be a real part of the name you asked for.

1.5.22 — Your turn is as short as possible. When an agent needs you (a sign-in, a 2FA code), it asks for that one step and does the rest itself. With done_when (what shows on screen once you're done), the prompt turns from blue back to amber the moment you finish, with no Resume click.

1.5.21

• set_value never guesses a field. With no field name or role it used to type into the window's largest text box; it now asks you to name the field.
• macOS: no screen-recording permission no longer breaks everything. Only screenshots and OCR fail, and they say how to fix it; windows, keys and accessibility keep working.
• Cleaner page text on Windows (no placeholder characters), and a security update to a dependency.

1.5.20

• The on-screen pill no longer blocks the agent's own clicks. It sits where browsers keep their tabs and address bar; it now steps aside for a moment before every click, drag or scroll, then comes back. Windows, macOS and Linux.
• Reading a page no longer highlights it on Linux and macOS either. copy_all_text reads the document through accessibility, as on Windows: exact text, nothing selected, your clipboard untouched.
• Your clipboard is put back after typing (smart_type too), and only once the paste has landed, so a slow app can't paste what you had copied.
• macOS: window focus focuses the process you asked for, a locked screen says so, sidebar clicks are verified, and a bare modifier key (shift) works.

1.5.19 — Typing never pastes your own clipboard. clawdcursor types by pasting; on Windows a clipboard write could fail silently and the paste then sent whatever you had copied. Every paste now checks the clipboard first and types key by key when it doesn't hold the text.

1.5.18

• "Something else" on the pause prompt. When the agent pauses for you, the prompt has a third button between Resume and End task: type an answer instead of doing the step — "use the test account instead", a code, a change of plan — and press Enter. The agent continues from what you wrote. Windows, macOS and Linux; over MCP pause_for_user returns {status:"replied", reply}.
• Reading a page no longer highlights it (Windows). copy_all_text reads the document the way a screen reader does: exact text, no Ctrl+A, nothing selected, your clipboard untouched.
• MCP tool annotations on every tool, and window / system now describe every action, its side effects and what it returns.

1.5.17

• macOS: text outside ASCII is typed and pasted as itself — "café 日本語" no longer comes out as "cafa aaa" or "caf√©".
• copy_all_text never selects an editor or a list (Ctrl+A there selected a whole document or every email).

1.5.16

• It pauses for you at a roadblock, then carries on. A sign-in, a 2FA code, a CAPTCHA, an upgrade page: instead of giving up, the agent pauses. The glow around the screen turns blue and an on-screen prompt says what it needs ("Sign in to Outlook, then click Resume"). Click Resume and it continues where it left off, or End task to stop. Your own agent can do the same over MCP with system({ action: "pause_for_user", message }); from a terminal, clawdcursor resume (or --end).
• The glow and banner on every OS. macOS and Linux now get the same screen-edge glow and banner as Windows: amber while the agent works, blue while it waits for you. Double-click the banner to stop everything.
• Blind mode on macOS caught up with Windows. A native accessibility helper lists windows in 0.3 s (was 11 s) and reads a window's tree in 0.06–0.2 s. Typing is refused while a system dialog owns the keyboard, and the agent is told when another app's dialog took its keys.
• Sharper blind mode on Windows and Linux. Classic controls no longer read as blank panes, the open dialog is what gets read, no 10 s stall on buttons that open a modal, the right field is found by name, and focusing a window no longer un-maximizes it.
• Upgrade walls are a roadblock. Clicks on Upgrade, Subscribe or Start trial need confirmation.
• Know what you're running. clawdcursor versions lists every installed copy (global npm, Claude Desktop extension, Claude Code plugin), and the installers say what they updated.

Everything else is in the changelog.

What it is

Clawd Cursor is a local MCP server that gives any tool-calling agent — Claude Code, Cursor, Windsurf, OpenClaw, the Claude Agent SDK, or your own loop — safe control of the real desktop. It clicks, types, reads the screen, opens apps, and drives any GUI the way a human would: native apps, the browser, even a canvas.

Most "let an agent use the computer" tools take a screenshot and feed it to a vision model — slow, expensive, and brittle. Clawd Cursor compiles the screen into one UI map: it fuses the accessibility tree and OCR into a confidence-scored set of elements, each tagged with a stable el_NN id, and acts on elements by id — not pixel coordinates. Coordinates appear only in the last-resort screenshot/vision tier (live pixels off the current frame), for canvas-only apps or tasks that genuinely need spatial reasoning. The result is cheaper, faster, private, and — uniquely — it checks that each action actually did what it claimed.

If a human can do it on a screen, your agent can too. No API, no integration, no problem — only the right sequence of reads, clicks, keys, and waits. Use it as the last-mile fallback: native API exists? Use it. CLI? Use it. Clawd Cursor is for the click, the legacy app, the GUI with no public surface.

Why it's different

The desktop-agent space is crowded. The closest install-and-go peers are Windows-MCP and Terminator (desktop MCP servers); browser-only tools (browser-use, Playwright MCP) are adjacent; and OmniParser / UI-TARS are vision-centric parsing approaches you'd build an agent around, not products you install. Here's the honest comparison across those approaches — what Clawd Cursor does that the popular options don't:

Three things here are genuinely rare:

1. Cheapest-tier-first perception, fully local. Accessibility tree (free) → OCR (cheap) → screenshot (expensive — the only tier that puts pixels in the model's context; "screenshot" and "vision" are the same step). The agent climbs only when it must, so token cost tracks task difficulty — and with a local model, nothing leaves the machine. Vision-centric agents (OmniParser, UI-TARS) need a screenshot in the model for every observation.