/clawdcursor

name: clawdcursor
version: 1.5.27
description: >
  FALLBACK ONLY — do not invoke unless you have already ruled out (1) a
  native API (Gmail API, GitHub API, Slack API …), (2) a CLI (git, gh,
  aws, npm, curl …), (3) direct file editing, and (4) existing browser
  automation (Playwright, Puppeteer). Only when all four are unavailable
  or have already failed should you use this skill. It gives AI agents
  a cursor and a keyboard on a real desktop — the last mile when the
  only remaining surface is a GUI. Concretely: use it when an earlier
  attempt via API, CLI, or direct file edit has failed and the user says
  things like "open X", "click Send", "type this in Word", "read what is
  on my screen", "do this in Outlook", "drive the Figma UI", "control my
  desktop", "automate this workflow", "fill out this form", or "copy text
  between apps". Works on Windows, macOS, and Linux with any LLM that can
  call functions (Claude, GPT, Gemini, Llama, Kimi, Ollama) over MCP —
  stdio for editor hosts (Claude Code, Cursor, Windsurf, Zed) or HTTP for
  daemons and dashboards.
homepage: https://clawdcursor.com
source: https://github.com/AmrDab/clawdcursor
privacy: >
  All processing runs locally. Server binds to 127.0.0.1 only - not
  network-accessible. No telemetry, no analytics. Screenshots stay in
  memory and go only to the MCP client that requested them; clawdcursor
  calls no model itself.
metadata:
  openclaw:
    requires: {}
    install:
      # Published on npm. (macOS also needs `clawdcursor grant` for the native
      # Accessibility/Screen-Recording helper; or use the install.sh one-liner.)
      - npm install -g clawdcursor
      - clawdcursor consent --accept
    skill_dir: ~/.openclaw/workspace/skills/clawdcursor

> **If you are a human:** see [`README.md`](README.md). This file is the AI-facing manual — dense on purpose. > > **If you are an LLM: keep going.**

---

> **FALLBACK ONLY.** Check these first — they're cheaper, faster, more reliable: > 1. Native API (Gmail, GitHub, Slack, Stripe …) → use the API. > 2. CLI (`git`, `gh`, `aws`, `npm`, `curl`) → use the CLI. > 3. Direct file edit → do that. > 4. Existing browser automation (Playwright, Puppeteer) → use that. > > **None apply? Now use clawdcursor.** It's the last mile — when the only remaining surface is a GUI. If a human can do it on a screen, you can too.

> **Capabilities & rules:** > - CAN: open apps, click, type, fill forms, read the screen (a11y tree / OCR / screenshot), drive browsers and native apps. > - MUST: before an irreversible action the user didn't ask for (send, delete, pay, close unsaved work), ask with `pause_for_user`. Ask before touching email, banking, messaging, or password managers. > - clawdcursor is your hands and eyes on the user's machine — used with their permission.

---

Start here - compact mode and the compound tools

> **If you are an LLM reading this: YOU SHOULD BE USING COMPACT MODE.** > > - MCP stdio (editor host): add `"args": ["mcp", "--compact"]` to your config. > - MCP over HTTP (daemon mode): start the daemon with `--compact` (or set > `CLAWD_MCP_COMPACT=1`) so `/mcp` serves the 6 compound tools (incl. `batch`). The surface is > fixed at startup — a daemon serves EITHER the compact tools OR the 100+ > granular ones, not both. Default (no flag) is granular. > > Granular mode's 100+ tools are kept for back-compat. Compact's tools are much smaller and reduce mis-tool-selection. Use granular only if your runtime MUST have every primitive as its own top-level schema.

**You drive the tools yourself. clawdcursor never runs a model.**

---

When you need step-level control - 6 compound tools

The compact surface collapses every primitive into five action-discriminated compound tools plus `batch`, mirroring Anthropic's `computer_20250124` pattern:

computer(action, ...)       Direct mouse / keyboard / screenshot / wait
accessibility(action, ...)  Read the a11y tree, click by name, set values, toggle
window(action, ...)         Open apps / focus / maximize / minimize / close / resize
system(action, ...)         Clipboard / time / OCR / undo / shortcuts / pause for the user
browser(action, ...)        DevTools Protocol - DOM-level control of any CDP-capable browser (Chrome, Edge, Chromium, Brave)
batch({steps})              Collapse N tool calls into one round-trip (see "Execution playbook" below)

Pick a compound FIRST based on what kind of operation it is, then set the `action` enum, then supply the args. The catalog is ~7,000 tokens - about a third of the granular surface's ~20,000 - so small models (Haiku, Kimi, Ollama) stay focused.

Cost tier - always use the cheapest tier that works

| Tier | Label | Cost | Use when | |---|---|---|---| | T1 | **structured** | ~free | Default. `accessibility.*`, `window.*`, `browser.read_text`, clipboard. Returns structured text — no image, no vision model. | | T2 | **OCR** | cheap | A11y tree is empty or sparse. `system {"action":"ocr"}`, plus `smart_read` / `smart_click` / `smart_type` — all OCR-backed (text out, no image into the model). | | T3 | **screenshot / vision** | expensive | A target with no name and no visible text (a drawing, a game, a chart), or a task needing spatial reasoning text can't express. `computer {"action":"screenshot"}` puts the current frame into the model's context; you then act on live pixel coords off that frame. "Screenshot" and "vision" are the same tier — the only one that sends pixels to the model. Last resort. |

**Rule: start at T1. Escalate to the next tier only when the current one fails.** Apply this logic when calling compound tools directly.

Quick reference - what action to pick

**I want to click something:**

• By name? → `accessibility {"action":"invoke","name":"Send"}`. Most reliable.
• By text via CDP on a web page? → `browser {"action":"click","text":"Submit"}`.
• Name not found (the label is drawn, not exposed)? → `accessibility {"action":"smart_click","target":"Send"}`