Gives AI agents control of a Linux desktop through accessibility trees, screenshots, window targeting, and synthesized click, type, and scroll input. Works through MCP, Pi tools, or a CLI.
Claude Code
> /plugin marketplace add agent-sh/computer-use-linux> /plugin install computer-use-linux@computer-use-linux
Codex
$ npx skills add agent-sh/computer-use-linux -a codex
Cursor
$ npx skills add agent-sh/computer-use-linux -a cursor
Gemini CLI
$ npx skills add agent-sh/computer-use-linux -a gemini-cli
GitHub Copilot
$ copilot plugin marketplace add agent-sh/computer-use-linux$ copilot plugin install computer-use-linux@computer-use-linux
OpenCode
$ npx skills add agent-sh/computer-use-linux -a opencode
Windsurf
$ npx skills add agent-sh/computer-use-linux -a windsurf
Running open models for your company? Tiyuvta helps with model choice, deployment and optimization, and fine-tuning on your hardware or cloud account.
computer-use-linux gives AI agents real, structured control of a local Linux desktop: semantic AT-SPI accessibility trees, bounded screenshots, compositor-aware window targeting, Wayland/X11 input synthesis, single-round-trip post-action verification, and a click-through on-screen activity indicator across GNOME, KDE Plasma/KWin, Hyprland, niri, COSMIC, i3, labwc, and X11/EWMH.
Pick the setup that matches your agent or shell workflow:
| Host / workflow | Install command |
|---|---|
| Claude Code (plugin + skill) | claude plugin marketplace add agent-sh/computer-use-linux && claude plugin install computer-use-linux@computer-use-linux |
| Codex CLI (plugin + skill) | codex plugin marketplace add agent-sh/computer-use-linux && codex plugin add computer-use-linux@computer-use-linux |
| Pi Coding Agent (native lazy-loaded tools) | pi install npm:@agent-sh/computer-use-linux |
| Hermes Agent (CLI + skill + MCP server) | npm install -g @agent-sh/computer-use-linux && hermes skills tap add agent-sh/computer-use-linux && hermes skills install agent-sh/computer-use-linux/computer-use-linux && hermes mcp add computer-use-linux --command computer-use-linux --args mcp |
| CLI + any stdio MCP client (npm) | npm install -g @agent-sh/computer-use-linux |
| CLI + system setup from source | git clone https://github.com/agent-sh/computer-use-linux && cd computer-use-linux && ./install.sh |
Rust crate (crates.io) | cargo install computer-use-linux |
Verify desktop readiness from the shell (after installing the CLI on PATH via npm install -g @agent-sh/computer-use-linux, ./install.sh, or cargo install computer-use-linux):
computer-use-linux doctor | jq .readiness
computer-use-linux setup # enable AT-SPI accessibility bridge if needed
computer-use-linux setup-window-targeting # install GNOME Wayland window-control extension if needed
Plugin-only or Pi-only installs: The Claude Code / Codex plugin launcher and
pi installmanage private cached binaries and do not addcomputer-use-linuxtoPATH. Either run the MCPdoctor/setup_accessibility/setup_window_targetingtools (orcomputer_use_linux_doctorin Pi) directly from your agent, or also install the CLI above when you want shell subcommands onPATH.
The Rust crate is published as computer-use-linux and the npm package as @agent-sh/computer-use-linux. Prebuilt x86_64 and aarch64 Linux binaries ship with the latest release. They need glibc 2.28 or newer: RHEL 8 and its rebuilds, Debian 10, Ubuntu 20.04, and anything later. On an older system, build from source with cargo install computer-use-linux.
Need an isolated desktop where the agent never touches your real mouse, keyboard, or focused window? Use the sibling project
agent-workspace-linuxalongside this one.
computer-use-linux is a Rust MCP server and CLI for Linux desktop control. It ships three binaries — the main computer-use-linux server/CLI, the computer-use-linux-cosmic Wayland toplevel helper for COSMIC, and the computer-use-linux-indicator click-through activity overlay. Any MCP host — Claude Code, Codex CLI, Codex Desktop's Linux build, Pi Coding Agent, Hermes Agent, Claude Desktop, or your own client — can spawn it to inspect accessibility trees, list and focus windows, take screenshots, click, drag, scroll, type, and invoke semantic accessibility actions.
Most computer-use MCP servers are macOS-only (they lean on AppKit, AXUIElement, CGEvent). The few that target Linux either drive xdotool against an X11 root window or shell out to OCR over screenshots. Five things set this one apart:
/dev/uinput or org.freedesktop.portal.RemoteDesktop on Wayland, with ydotool / ydotoold as the fallback. Literal text prefers KDE clipboard integration on Plasma, wtype on compatible Wayland compositors, and portal keysyms on other Wayland sessions, including GNOME with ydotool installed. Portal text startup or conversion errors fail the call without replaying text through ydotool. Raw ydotool text accepts printable ASCII, tab and newline only, and its physical US key positions can produce different text under the active layout. Screenshots use the GNOME Shell DBus screenshot method when present, org.freedesktop.portal.Screenshot otherwise, then on a native X11 session a root-window GetImage (device pixels, no toolkit scaling), and finally spawn gnome-screenshot for background/systemd contexts where the DBus paths are denied.hyprctl, the niri IPC socket, i3 IPC, labwc's labwc.sock, and generic X11/EWMH in order, then reports exactly which backend won or why each backend failed. Terminal windows are enriched with their controlling TTY and foreground process so agents can target "the terminal running pytest" directly. Every window_id fits JavaScript's exact integer range (at most 2^53-1), so JSON clients can pass it back unchanged; a larger value is rejected with a prompt to re-run list_windows.click, perform_action, and set_value accept role / name / text / states selectors or element_index values backed by AT-SPI, and every mutating tool can return a post-action screenshot and updated accessibility tree in the same round trip. Pixel coordinates remain available for rendering-only surfaces (canvas, games, X clients without ATK).computer-use-linux-indicator renders a click-through overlay with a per-agent cursor glide, click ripple, keycaps/typing banner, screen-edge glow, and status pill — and hides automatically under a shared capture lock before any screenshot is read.computer-use-linux doctor returns a structured document covering platform, portals, AT-SPI, windowing, input, and a readiness summary with explicit blockers and a recommended next step. MCP hosts can render or surface that to the user without parsing prose.The crate was extracted from codex-desktop-linux (the Linux distribution of Codex Desktop), which bundles this backend for its native Computer Use feature. This standalone repo is the upstream.
| Category | Tool | What it does |
|---|---|---|
| Diagnostics & setup | doctor | Single-shot JSON readiness report (platform, portals, accessibility, windowing, input, readiness summary, and capability map). |
setup_accessibility | Enables and verifies GNOME's org.gnome.desktop.interface toolkit-accessibility setting so toolkit apps expose AT-SPI trees. | |
setup_window_targeting | Installs and enables the bundled GNOME Shell extension when org.gnome.Shell.Introspect is locked down. | |
| Discovery & observation | get_app_state | Combined screenshot + compacted AT-SPI accessibility tree with stable element_index handles. Scope with app_name_or_bundle_identifier or a window target (window_id, pid, app_id, wm_class, title). |
get_tree_page | Page through the latest cached tree by node offset, for trees too large to return inline when the host cannot read the spill file. | |
screenshot | Bounded PNG or JPEG capture of the full desktop (full_screen: true or no target) or a raised and cropped target window (window_id, pid, app_id, wm_class, title, raise_window). | |
list_windows | Compositor windows with window_id, title, app_id, wm_class, PID, focus state, client type (wayland/x11), bounds, and terminal TTY/process metadata. | |
focused_window | The window currently holding keyboard focus. | |
list_apps | Running desktop applications visible to the AT-SPI registry. | |
| Input | click | Click by element_index, semantic selector (role/name/text/states), or desktop/window-relative coordinates (left/right/middle, click_count). |
drag | Desktop coordinate drag from (start_x, start_y) to (end_x, end_y). | |
scroll | Page-based vertical/horizontal scroll on an element, window, or pixel coordinate. | |
press_key | Single keys or chords (Ctrl+L, Alt+F4, Return), optionally focusing a target window or terminal first. | |
type_text | Literal text entry, optionally targeted at a window or terminal (tty, terminal_pid, terminal_command, terminal_cwd). | |
| Semantic actions | perform_action | Invoke any AT-SPI action exposed by an element (Press, Activate, Toggle, Jump, …); defaults to the primary action. |
set_value | Write directly to a settable accessibility element (text fields, sliders, spinners). | |
| Window management | activate_window | Focus a window by window_id, pid, app_id, wm_class, title, or terminal selectors. |
move_window / resize_window | Reposition or resize a window in desktop coordinates (GNOME Shell extension and X11/EWMH backends). | |
| Opt-in host tools | complete_interaction | Parameter-free desktop completion notification (notify-send); registered only when COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE=1. |
run_shell | Bounded non-login /bin/sh -c execution; registered only when COMPUTER_USE_LINUX_ENABLE_SHELL=1 (absent by default). |
click, scroll, perform_action, and set_value take an optional app target (pid, window_id, app_id, wm_class, window_title). With one, an element_index or selector from another app's get_app_state snapshot is refused; call get_app_state for the target app first. When the window backend reports no pid for a target window (i3, sway, COSMIC), the snapshot must have been taken for that window, or the call is refused. perform_action and set_value do not focus the target window. An element_identifier is refused when its owner on the accessibility bus is a different pid than the target; it is not checked when either pid is unknown.
Stars on GitHub
View @agent-sh’s profileA runtime that runs the work around AI-written code for you: picking tasks, managing branches, reviewing code, cleaning up artifacts, handling CI and PR comments, and deploying. Pipelines pause at gates and keep state across sessions.
@agent-shLints and validates AI coding assistant configuration files such as CLAUDE.md, AGENTS.md, SKILL.md, hooks and MCP setups, with autofixes and editor integration for common IDEs.
@agent-shFair-code platform to build and deploy AI agents and workflows. Combine a visual canvas with custom code, run it self-hosted or in the cloud, and connect to 1500+ integrations. AI automation you can trust with real work, from prototype to production.
@n8n-ioA browser automation plugin for Claude Code with skills for direct browser control, scraping, testing, and QA scoring of websites, plus guidance on the Browser Use cloud browser API and Python library.
@browser-useOrganizes your AI agents into a team that runs around the clock, handling hiring, scheduling, delegation and reporting so you can stay in charge without staying online.
@lobehubA browser automation command line tool for AI agents, letting them navigate pages, fill forms, click buttons, take screenshots and extract data from websites.
@vercel-labs-2Gives AI agents control of a Linux desktop through accessibility trees, screenshots, window targeting, and synthesized click, type, and scroll input. Works through MCP, Pi tools, or a CLI. Install it with the command on its page. It includes computer-use-linux. Its skills do not fire on their own yet. Free and open source.
Is this plugin yours?
Claim it with GitHubSubmit a pluginPromote it