/jev-desktop

name: jev-desktop
description: >
  Drive a desktop application from one plain-language goal, or resolve one intent
  at a time, without ever reading the accessibility tree. Use when automating a
  desktop app and you want to keep a 150-element JSON tree out of your context.
  Hand `run.mjs` a whole goal and it observes, decides and acts until the goal is
  met; hand `act.mjs` a single step when you want to keep the plan yourself.
  Covers which element, which operation, when to look deeper, and when to stop.
  Needs a clone of the agent-desktop repository and a TYPESAFE_API_KEY.

jev-desktop

This skill requires a clone of the agent-desktop repository (the scripts live in `scripts/jev/` and ship in no installed package) and a `TYPESAFE_API_KEY`.

Two entry points over the same screen reading. `run.mjs` takes a goal and drives until it is met. `act.mjs` takes one step and hands the plan back to you. Neither puts the tree in your context.

The loop

node scripts/jev/run.mjs --app Finder "open the Applications folder"
node scripts/jev/run.mjs --app TextEdit --cursor --text "A line to write." "write the supplied sentence into the document"

Each turn reads the screen, asks for one operation and a target for that operation in the same request, carries it out, and reads again. It prints one JSON line per turn and one when it stops (shown trimmed).

{"turn":{"step":1,"operation":"CLICK","target":"treeitem \"Applications\"",
         "confidence":1,"ok":true,"delivery":"delivered_verified","changed":true}}
{"stop":"done","confidence":0.96}

The operations are `CLICK`, `TYPE_TEXT`, `CHECK`, `UNCHECK`, `EXPAND`, `COLLAPSE`, `SCROLL`, `DRILL`, `WIDEN`, `WAIT`, `DONE` and `BLOCKED`. Only the ones something on screen can receive are offered: `DRILL` for a region that reports it holds more, `WIDEN` only after a `DRILL`. `SCROLL` always scrolls down. `delivery` is `waited` for `WAIT` and `looked` for `DRILL` and `WIDEN`.

**The target is asked once per operation.** `click_target` is chosen from the elements that advertise `Click`, `type_text_target` from those that advertise `SetValue`, and so on. Only the head matching the chosen operation is read, so a target can never be incompatible with the verb that acts on it.

**Looking is an operation.** A window is read with `--skeleton` first, so a dense document costs the same first look as a small panel. A region that was cut off reports how much it holds, and `DRILL` pins it as the root for later turns. `WIDEN` gives the whole window back.

**`CHECK` and `UNCHECK` instead of a toggle.** Both are idempotent: a box already in the wanted state stays there.

**You supply the text.** Nothing here writes a value, so a run never puts a string on screen that you did not choose. Pass `--text` once per value and they are consumed in order, or pass a function as the `text` option and it is asked for each field with that field's description. When there is no value left to give, `TYPE_TEXT` is not offered at all, so the run stops rather than inventing one.

**Text goes in by whichever route the application accepts.** A direct value write (`set-value`) is one verified call. Only when the application refuses it and the refusal says a retry is safe does the value go through the clipboard and a `cmd+v` paste (macOS only). The clipboard is put back when the run ends, or emptied if it held anything but text. One key press per character is never used: it drops characters and loses capitals.

**What leaves the machine.** Every turn posts a description of the screen to the decision endpoint (TypeSafe's `api.typesafe.ai` by default, or whatever `TYPESAFE_BASE_URL` points at): each element's role, its accessible name or description, up to sixty characters of the value it holds, its state, the window title, and the recent actions, excluding supplied text and detailed execution evidence. That is enough to send the contents of a private document or a filled form to the configured remote service. Secure text fields are already withheld by agent-desktop and never reach the request. Pass `--no-values` to either script to withhold what every other field holds as well; targeting gets harder, because a value is often the only thing that tells two unnamed rows apart. The default is to include values. `run.mjs` redacts explicit supplied strings from action turns and later recent-action history, including when later target names or execution evidence repeat them. Live screen descriptions remain separate: if the application displays that text, its accessible name or field value may still reach the remote service under the settings above. `act.mjs` preserves its exact executable `argv` in stdout, so that field can include a value supplied with `--text`; stdout may also contain accessible names and element paths. Choose the endpoint and output destination with that exposure in mind.

**A step that is hard to undo is not taken quietly.** An ordinary step needs 0.70 confidence in its target, one rated destructive (0.50 or more) needs 0.90, and below 0.55 nothing runs. When the bar is not met the run stops and names the candidate it would have acted on, so you decide instead of it. `act.mjs` gets the rating in its single request. `run.mjs` rates risk with a second model call (counted toward the 80), only for targets at 0.70 up to 0.90, so by default a destructive step at target confidence 0.90 or higher runs unrated. If that call returns no number, the step counts as destructive. Pass `--confirm-destructive` to `run.mjs` or `act.mjs` to stop for confirmation on every step rated destructive, regardless of confidence.

**`--cursor` makes the run watchable.** It starts a session, shows a cursor that travels to each element before the operation lands, and turns it off at the end. If the session cannot start, the run continues without a cursor. The cursor is drawn where the element is, so bring the application in front of your terminal or it arrives behind it.

It st