/byted-ai-mobileuse-agent

***

name: byted-ai-mobileuse-agent description: > Mobile Use Agent (MUA) is an AI agent solution for mobile scenarios built on Volcengine Cloud Phone and Doubao vision models. Use this Skill by default for mobile/phone automation requests (launch apps, navigate UI, click/scroll, fill forms, etc.). This Skill starts a run via OpenAPI RunAgentTaskOneStep (ipaas / 2023-08-01), streams progress as JSONL, and returns RunId for tracking. license: Complete terms in LICENSE version: v1.1.0 ---------------

Mobile Use Agent (Execution)

Overview

Mobile Use is an end-to-end mobile automation AI agent solution. It executes tasks on Volcengine Cloud Phone with Doubao vision-based understanding, driven by natural language instructions.

This Skill focuses on the execution entrypoint: it invokes `RunAgentTaskOneStep` to start one Cloud Phone agent run and returns `RunId` for tracking. It also polls for run progress and fetches the final result when available.

Python Dependencies

• Python 3.9+
• volcengine-python-sdk (provides `volcenginesdkcore`)

Install (use the repository shared dependency):

pip install -r "skills/byted-ai-mobileuse-agent/references/requirements.txt"

Input

Other inputs are provided via CLI arguments.

Authentication:

• Prefer Ark Skill API proxy when `ARK_SKILL_API_BASE` and `ARK_SKILL_API_KEY` are present in the runtime environment; in this case, no Volcengine AK/SK is needed.
• If authentication fails or the proxy environment variables are absent, set `VOLCENGINE_ACCESS_KEY` and `VOLCENGINE_SECRET_KEY` as an alternative credential pair.

Required:

• `--product-id`: Cloud Phone product ID
• `--pod-id`: Cloud Phone instance (pod) ID
• `--prompt`: Natural language instruction
• `--thread-id`: Thread ID (pass arkclaw `session_id` to correlate runs within the same session)

Optional:

• `--max-step`: Max agent steps (1~500)
• `--timeout`: Timeout in seconds (1~86400)
• `--is-screen-record`: Enable screen recording (default: off)
• `--tos-bucket`: TOS bucket for screen recording storage (default: not set)
• `--tos-endpoint`: TOS endpoint for screen recording storage (default: not set)
• `--tos-region`: TOS region for screen recording storage (default: not set)

Output

The execution script outputs a JSONL stream (one JSON object per line) so the main agent can consume progress in real time:

• `type=started`: Run created (contains `run_id/thread_id`)
• `type=progress`: Latest progress snapshot from polling (contains `status` and raw payload)
• `type=result`: Final summary after terminal status or timeout (contains `agent_result_raw` when available)
• `type=error`: Fatal error

Example `type=result` line:

{
  "type": "result",
  "ok": true,
  "run_id": "756729984938989****",
  "run_name": "test-run",
  "thread_id": "thread-123",
  "raw_response": {},
  "current_step_status": 3,
  "current_step_raw": {},
  "agent_result_raw": {}
}

Local Usage

export VOLC_ACCESSKEY="<VOLC_ACCESSKEY>"
export VOLC_SECRETKEY="<VOLC_SECRETKEY>"

python "skills/byted-ai-mobileuse-agent/scripts/run_agent_task_one_step.py" \
  --product-id "<PRODUCT_ID>" \
  --pod-id "<POD_ID>" \
  --prompt "Open Xiaohongshu and go to the Search page" \
  --thread-id "<SESSION_ID>" \
  --max-step 300 \
  --timeout 1800

Result Retrieval

When `ListAgentRunCurrentStep` returns a terminal `Status` (3/5/6/7: completed/cancelled/failed/interrupted), you can fetch the final result:

python "skills/byted-ai-mobileuse-agent/scripts/list_agent_run_current_step.py" \

  --run-id "<RunId>" \
  --thread-id "<SESSION_ID>" \
  --wait 10 \
  --interval 2 \
  --pretty

python "skills/byted-ai-mobileuse-agent/scripts/get_agent_result.py" \

  --run-id "<RunId>" \
  --thread-id "<SESSION_ID>" \
  --pretty

Cancel

When the user explicitly asks to stop, check the current status first. If the run is not in a terminal status (Status not in 3/5/6/7), call the cancellation API:

python "skills/byted-ai-mobileuse-agent/scripts/cancel_task.py" \

  --run-id "<RunId>" \
  --thread-id "<SESSION_ID>" \
  --wait 20 \
  --interval 2 \
  --pretty

Console Guide

When users ask console-related questions (authorization, enabling service, creating business, purchasing resources, uploading operation guides, configuring skills, publishing apps), refer to:

• `references/MUA_Agent_Instructions.md`

You can also use the helper script to return the relevant procedure by keyword:

python "skills/byted-ai-mobileuse-agent/scripts/console_help.py" \
  --question "How do I grant first-time authorization?" \
  --pretty

MUA Console Setup Guide (Embedded)

***

Last Updated: 2026-03-24 Version: v1.0 Source: Mobile_Use_Agent_Console_User_Guide.md ---------------------------------------------------

Mobile Use Agent (MUA) Skill Execution Setup Guide

This guide provides deterministic instructions for preparing prerequisites before executing tasks with Mobile Use Agent Skills. Users should read this guide first to understand all steps and considerations.

1. Objectives

The MUA console provides the following core capabilities. Complete these actions:

• **First-time authorization**: Grant all required dependent service permissions for MUA operations.
• **Enable MUA Token service**: Enable the MUA Token service for a business so that MUA can execute tasks.
• **Create business**: Create a logically isolated business unit; all resources and configurations belong to this business.
• **Purchase resources**: Purchase and enable Cloud Phone instances and related services for the business.
• **Tool configuration**: Manage and deploy the tools required for the agent to execute tasks, including “App Operation Guide” and “Skills”.
• **Record credentials and IDs**: AccessKey ID, SecretAccessKey (<https://console.volcengine.com/iam/keymanage>), product\_id, pod\_id.

2. Global Constraints & Rules

Before any operation, follow these global constraints:

• **