/argent-device-interact

name: argent-device-interact
description: Interact with an iOS simulator or iPhone, Android emulator or phone, or Chromium (CDP) app using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling/swiping, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots, waiting for an element to appear or disappear, or checking visible app state after interactions. Not for TV targets.

Each tool's own description carries its parameters, defaults, return shape and examples. This skill covers choosing between tools and the gotchas those descriptions don't state.

1. Before You Start

• Get a target id from `list-devices`; if none is ready, `boot-device` one. Full setup flow: `argent-ios-simulator-setup` / `argent-ios-device-setup` / `argent-android-emulator-setup`. The same tool names drive iOS, Android and Chromium; each takes the `udid` from `list-devices`.
• Physical iPhone (`kind: "device"`): read `argent-ios-device-interact` first. Only the tools it lists exist there, and `describe` sees only the app `launch-app` registered.
• Mac app (`platform: "macos"`): read `argent-macos-app-interact` first.
• Read the matching reference before driving these targets: `references/chromium.md` for a `chromium` target, `references/harmonyos.md` for a `harmony` target, `references/foldables.md` for a device `list-devices` marks `foldable: true` (or before calling `fold`).
• Load a deferred tool's schema with ToolSearch before its first call (e.g. `gesture-pinch`, `gesture-rotate`, `gesture-custom`); without it, numbers go out as strings and fail validation.
• If you delegate device tasks to sub-agents, make sure they have MCP permissions.

2. Choosing the Right Tool

| Action | Tool | Notes | | ------------------------------ | ------------------- | -------------------------------------------------------------- | | Several known steps | `run-sequence` | One call, one screenshot at the end (§5) | | Open an app | `launch-app` | **Never tap home-screen icons** | | Restart an app | `restart-app` | Clean in-memory state | | Open URL / deep link | `open-url` | Web pages, deep links, URL schemes | | Tap | `gesture-tap` | Buttons, links, checkboxes | | Scroll / swipe | `gesture-swipe` | Touch targets; not Chromium | | Scroll (Chromium) | `gesture-scroll` | Mouse wheel | | Drag (Chromium) | `gesture-drag` | Sliders, drag-and-drop, text selection | | Long press, drag & drop | `gesture-custom` | Also any other non-linear touch sequence | | Pinch / zoom | `gesture-pinch` | | | Two-finger rotation | `gesture-rotate` | | | Hardware key | `button` | Home, back, power, volume, appSwitch, actionButton | | Type text or press a named key | `keyboard` | Every platform; text or one key per call (§6) | | Paste text | `paste` | Only where a user would paste (§6). Sim/emu, Mac app | | Rotate device | `rotate` | Orientation changes | | Fold device | `fold` | Foldables only: `references/foldables.md` | | Shake device | `shake` | Shake handlers (sim/emu only), Undo-typing prompt, RN dev menu | | Wait for UI | `await-ui-element` | Block until an element is visible/hidden/exists/contains text | | Wait for idle | `await-screen-idle` | Block until a non-empty screen tree stops changing (§4) |

3. Finding Tap Targets

After an action moves you to a different screen, or whenever you don't know an element's coordinates, discover before tapping. Interaction results already carry the element tree (`--- Elements after action (describe) ---`): use it, and call a discovery tool only when no fresh tree covers the current screen.

| Need | Tool | | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | Any screen: app, system dialog, Home screen | `describe` | | UIKit properties of a view (iOS simulator) | `native-find-views` (needs `bundleId` and a class, id, label, tag or `nativeID` to match) | | What sits at a known raw point (iOS simulator) | `native-user-interactable-view-at-point` (the view that receives touch) or `native-view-at-point` (the visually deepest view) | | Final visual fallback | `screenshot`, only when the tools above cannot expose the control. Do not derive routine in-app navigation targets from pixels |

If `describe` returns an empty tree, follow its `hint`; with no hint, the screen is blank, loading or unlabeled: check the attached screenshot and retry once content has loaded.

For a React Native testID or component name that `describe` does not show, use `debugger-componen