/agent-desktop-ffi

name: agent-desktop-ffi
tags: ffi, c-bindings, cdylib, python, swift, node, go, rust-ffi
requirements:
  - agent-desktop-ffi
description: >
  C-ABI bindings over agent-desktop's PlatformAdapter. Consumers
  (Python ctypes, Swift, Node ffi-napi, Go cgo, C++, Ruby fiddle)
  link libagent_desktop_ffi.{dylib,so,dll} and call `ad_*` functions
  directly instead of spawning the CLI binary per call. The canonical
  observe-act workflow is: ad_init → ad_adapter_create[_with_session]
  → ad_snapshot → read qualified refs (@s8f3k2p9:e5) → ad_execute_by_ref → ad_free_string
  → ad_adapter_destroy.

agent-desktop-ffi

Direct C-ABI access to PlatformAdapter operations. Build the cdylib with the `release-ffi` profile:

cargo build --profile release-ffi -p agent-desktop-ffi

The output is `target/release-ffi/libagent_desktop_ffi.dylib` (`.so` on Linux, `agent_desktop_ffi.dll` on Windows) plus a committed C header at `crates/ffi/include/agent_desktop.h`. The default `release` profile uses `panic = "abort"`: a Rust panic inside an `extern "C"` function would `SIGABRT` the host. `release-ffi` keeps `panic = "unwind"`, which the panic traps need.

Four reference topics, loaded as needed:

• [ownership.md](references/ownership.md): who allocates and who frees every

pointer the FFI hands back.

• [error-handling.md](references/error-handling.md): errno-style last-error

contract, error codes, enum validation, panic boundary.

• [threading.md](references/threading.md): host-thread contract, mutation

ordering across threads and processes, adapter-bound native handles, AXIsProcessTrusted inheritance.

• [build-and-link.md](references/build-and-link.md): prebuilt archives, ABI

handshake, struct size validation, C and Python examples.

Observe-act workflow (canonical path)

ad_init(AD_ABI_VERSION_MAJOR)                    // verify header ↔ dylib match
adapter = ad_adapter_create_with_session("s1")   // or ad_adapter_create()
rc = ad_snapshot(adapter, "Finder", 0, 10, false, false, &json_out)
// parse json_out: locate snapshot-qualified refs in data.tree
ad_free_string(json_out)
// build action:
AdAction act = {0}; act.kind = AD_ACTION_KIND_CLICK;
rc = ad_execute_by_ref(adapter, "@s8f3k2p9:e5", &act, 0, &result_out)
ad_free_string(result_out)
ad_adapter_destroy(adapter)

`ad_snapshot` returns the CLI envelope `{version, ok, command, data}`. `data.tree` carries snapshot-qualified refs for interactive elements. Read `data.complete` before trusting a tree: `false` means the observation budget ran out and the tree is partial, not an error. `ad_snapshot` always targets the focused window of `app`; window targeting, skeleton mode and `--root` drill-down are not exposed over the ABI. A bare `@eN` ref returns `AD_RESULT_ERR_INVALID_ARGS`.

Core constraints

• **ABI handshake.** Call `ad_init(AD_ABI_VERSION_MAJOR)` once after loading the

library. A mismatch returns `AD_RESULT_ERR_INVALID_ARGS`: abort. `ad_abi_version()` returns the library's major for display. The major increments on a removed symbol or a layout change; new `ad_*` symbols and error codes are additive. Pin the exact `libagent_desktop_ffi` version you link against.

• **Session adapters.** `ad_adapter_create_with_session("session-id")` scopes

refmap persistence to a session namespace, like CLI `--session <id>`. IDs are 1 to 64 ASCII alphanumeric, `-`, or `_` characters. A null ID behaves like `ad_adapter_create()`. An invalid ID returns null (read `ad_last_error_*`).

• **Ref actions.** `ad_execute_by_ref` is the primary action entrypoint. It loads

the RefStore, resolves the ref strictly (`STALE_REF` when the snapshot has no such ref or the element changed, `AMBIGUOUS_TARGET` on several candidates), runs the live actionability preflight (visible, stable, enabled, supported action, policy, editable), then dispatches. A disabled or unsupported target fails before any platform call; on `AD_RESULT_ERR_ACTION_FAILED` the check report is JSON in `ad_last_error_details()`. It auto-waits 5000 ms before the preflight; `ad_execute_by_ref_timeout` takes `timeout_ms` (`-1` = 5000 ms default, `0` = single-shot). `PressKey` defaults to `focus_fallback`; every other action, including `TypeText`, defaults to `headless`. Pass `AD_POLICY_KIND_FOCUS_FALLBACK` (1) to allow focus without cursor movement, or `AD_POLICY_KIND_HEADED` (2) to allow physical cursor and keyboard delivery. `ad_execute_ref_action_exact_with_policy` runs the same preflight from an `AdExactRefEntry`.

• **Generation-safe direct APIs.** Every direct targeting function takes

process-generation evidence: `AdExactRefEntry`, `AdExactWindowInfo`, `ad_list_windows_exact`, `ad_launch_app_exact`, and the other `*_exact` symbols. `AdRefEntry`, `AdWindowInfo`, and `AdSurfaceInfo` exist only as fields embedded in the exact structs. `ad_list_surfaces_exact` keeps the surface `id`. From `AD_EXACT_SURFACE_INFO_VERSION` 2, `AdExactSurfaceInfo.unclassified` names the surface kinds its probe could not read (for example `sheet,menu`), or is null. A missing `sheet` or `menu` next to a listed kind is unknown, not absent. Only the Windows adapter fills it; macOS and Linux return null.

• **Native handles.** `ad_resolve_element_exact` and `ad_find_exact` return an

`AdNativeHandle`. `ad_execute_action` (headless, no preflight) and `ad_execute_action_with_policy` act on it directly, bypassing the ref-action pipeline. Release every handle with `ad_free_handle(adapter, &handle)` on the same adapter and thread, before `ad_adapter_destroy`, on every platform. It zeroes `handle.ptr`, so a second call is a no-op.

• **Action results.** `AdActionResult.steps` mirrors the CLI `steps` array for

activation-chain actions. Each `AdActionStep` has `label` and `outcome` strings, `mechanism` (`AD_STEP_MECHANISM_SEMANTIC_API` = 1 or `AD_STEP_MECHANISM_PHYSICAL_SYNTHETIC` = 2, valid only when `has_mechanism`), and `verified` (valid only