Does opengui auto-invoke its skills?

Not yet. It is indexed on Flowy as a plain plugin. Request auto-invocation on its page and Flowy will route its skills for you as you prompt.

Yes. Flowy is free and open source, with nothing gated. You can read every skill in full before you install.

Automation

Plugin

opengui

OpenGUI is an Android GUI agent framework for phone-use AI that can see, plan, and operate real mobile apps through the GUI.

FLOW.md

core-mate@core-mate

GitHub

Plugin Signal65/100Solid

maintained81

capable23

documented100

adopted68

Install

$ npx -y skills add Core-Mate/OpenGUI --agent claude-code

Run the curl in your terminal, the rest in Claude Code.

Stats

TypeScript Bash Java Kotlin PostgreSQL

+7 tags

Redis REST API Docker Mermaid Anthropic OpenAI Android

1,534

Stars

Forks

Maintained

Maintenance

Kotlin

Language

1mo ago

Last commit

3mo ago

Created

Repo: Core-Mate/OpenGUI

In this plugin

2 skills

- open-gui-bootstrap
- open-gui-remote-control

See everything inside

What's inside

Skills

/open-gui-bootstrapLaunch and bootstrap OpenGUI from a plain-language user request. Use when Claude or Codex should install, configure, debug, and run the actual backend and Android client shipped in this repository while keeping manual setup to a minimum./open-gui-remote-controlControl an Android phone through OpenGUI from Codex or Claude Code. Use when an agent should list online devices, run a natural-language mobile task, check execution status, pause, resume, or cancel through the local OpenGUI backend and CLI.

README

Read on GitHub

Demo

OpenGUI reads a real Android app UI, plans the next step, takes mobile actions, and returns structured results.

Quick Start

The fastest way to try OpenGUI is to let Claude Code or Codex bootstrap it for you.

Read ./skills/open-gui-bootstrap/SKILL.md and help me run OpenGUI. Only ask me for phone-side actions.

You will need:

an Android phone or emulator
USB debugging enabled
AccessibilityService enabled
model API keys for real task execution

OpenGUI will use the repository scripts to start the backend and install the Android client:

cd server
./start.sh

cd client
./start.sh

After the backend and Android client are running, send a first task:

cd server
pnpm opengui -- devices --json
pnpm opengui -- do "Observe the current Android screen and summarize what you see" --json

Manual setup guide: docs/get-started.md.

Recent Updates

[2026.5.16] Added Codex / Claude Code remote control with a local REST API, pnpm opengui -- ... CLI, and the open-gui-remote-control Skill for dispatching Android app tasks from coding agents.
[2026.5.9] Added a Discord IM channel for remote Android task dispatch, including prefix commands, slash commands, allowlists, and guild-scoped command registration.
[2026.5.7] Hardened local startup to avoid common PostgreSQL and Redis port conflicts during Docker-based backend setup.
[2026.5.1] Improved backend onboarding with .env.example, startup checks, and graph-agent VLM environment configuration.

What You Can Do with OpenGUI

OpenGUI provides an Android GUI agent stack for screen understanding, task planning, action execution, review, and recovery.

You can use the same repository in four practical ways:

Operate mainstream Android apps: let AI handle mobile tasks inside X, Reddit, Hacker News, Telegram, WeChat, Weibo, Xiaohongshu, and other Android apps on a real phone.
Run shipped workflows: the repository already includes a runnable backend, Android client, standby dispatch path, and a set of built-in task capabilities.
Let Claude or Codex bootstrap it for you: point the model at skills/open-gui-bootstrap/SKILL.md, describe the goal in plain language, and let it handle setup, build, install, and local debugging.
Let Codex control Android apps: after OpenGUI is running, point Codex or Claude Code at skills/open-gui-remote-control/SKILL.md to list devices, dispatch tasks, and track executions through the local CLI.
Operate phones as remote workers: dispatch tasks through Feishu, Telegram, Discord, or REST API, keep devices on standby, and get structured results back from the backend.
Join the Discord community

Highlights

Built for long-running tasks: OpenGUI is shaped for mobile workflows that may run for hours, with progress, review, and recovery kept inside the system.
Plan before action, summarize after execution: before touching an app, OpenGUI breaks the goal into executable steps; after the run, it returns a structured summary of what happened, what worked, and what still needs attention.
The task can keep moving: Plan Supervisor maintains task state and continuation, Executor Graph runs screenshot, vision, action, and call-user loops on top of live device state, and Summarizer closes the run with a structured result.
Phones can stay on standby: the standby dispatch path lets devices receive remote work through Feishu, Telegram, Discord, or REST entry points.
Models can be assigned by role: model routing separates planning from VLM execution so teams can choose providers by job.
The system is organized around real mobile workflows: the graph, device execution path, and model split already exist in the source tree.

Why OpenGUI Is Different

OpenGUI is built as a mobile operator system with explicit orchestration layers.

The source code currently exposes these pieces:

server/apps/backend/src/modules/graph-agent/graph/mobile-agent.graph.ts for the main graph
server/apps/backend/src/modules/graph-agent/graph/executor.graph.ts for the device-side execution loop
server/apps/backend/src/common/ws/standby.gateway.ts for standby device dispatch
client/core_network/.../StandbySocketManager.kt for persistent device standby connections
client/core_accessibility/.../GestureService.kt for Android-side action execution

Dimension	Typical phone-agent demo	OpenGUI
Execution model	Short interactive loop	Main graph plus executor subgraph
Task state	Usually local and session-bound	Task state managed in the backend graph
Device path	Often laptop-driven control	Android client with standby and execution sockets
Model usage	One model does most of the work	Planning and VLM paths can be split across providers
Remote operation	Optional add-on	Feishu, Telegram, Discord, REST API, and standby dispatch are built into the backend

Typical Use Cases

Open X and collect recent posts for a topic
Read and summarize Reddit or Hacker News threads on a live phone
Trigger Android tasks remotely from Feishu, Telegram, Discord, or REST API
Execute repetitive mobile workflows on Android devices
Run long mobile workflows that need state, review, and recovery over many hours

Current Limitations

Requires an Android device or emulator.
Requires USB debugging and AccessibilityService permissions.
Execution quality depends on the model, app UI, network state, and task length.
Not an always-on OS-level assistant yet; tasks are currently triggered manually or through configured dispatch channels.
Long-running tasks are supported by the system design, but reliability still needs more real-world testing.
More ready-to-run task examples and benchmarks are still needed.

Roadmap

Add a short demo video and more real app examples.
Improve one-command local setup.
Add more ready-to-run phone-use task templates.
Improve execution recovery and failure reporting.
Add benchmark tasks for Android GUI agent reliability.
Expand docs for model configuration and cost-saving profiles.

How to Use OpenGUI

1. With Claude or Codex

Start with skills/open-gui-bootstrap/SKILL.md.

The intended flow is simple:

point Claude or Codex at the skill
describe the task in plain language
let the model handle backend bootstrap, APK build, install, and local debugging

It should only stop for:

connecting a phone or starting an emulator
approving USB debugging
enabling AccessibilityService
granting overlay or battery permissions
providing API keys or bot credentials

After the backend and Android client are running, use skills/open-gui-remote-control/SKILL.md to let Codex or Claude Code control the phone through the local CLI:

cd server
pnpm opengui -- devices --json
pnpm opengui -- do "Observe the current Android screen and summarize what you see" --json
pnpm opengui -- status <executionId> --json

Recommended profiles:

High-performance profile

Use the latest Claude Opus model family across planning, supervision, review, and vision when you want the strongest overall quality.

This is the easiest way to get the best execution quality, and it is the most expensive path.

Cost-saving mixed profile

Use Qwen 3.6 Plus for text-side roles such as Planner and Supervisor, and use Doubao Pro for the VLM side.

This usually preserves the overall system shape while lowering model cost by roughly 10x to 15x compared with an all-Opus setup, depending on task length, screenshot volume, and token mix.

Recommended prompts:

Run it

Read ./skills/open-gui-bootstrap/SKILL.md and help me run OpenGUI. Only ask me for phone-side actions.

Use Claude Opus everywhere

Read ./skills/open-gui-bootstrap/SKILL.md and bootstrap OpenGUI with the latest Claude Opus model family for planning, supervision, review, and vision.

Use Qwen + Doubao to save cost

Read ./skills/open-gui-bootstrap/SKILL.md and set up OpenGUI with Qwen 3.6 Plus for Planner and Supervisor, and Doubao Pro for VLM execution.

Use my own APIs

Read ./skills/open-gui-bootstrap/SKILL.md and use my existing model APIs to get OpenGUI working.

2. Manual setup

Use the repository scripts directly:

cd server
./start.sh

cd client
./start.sh

Reference docs:

3. Optional Discord remote control

Discord can be enabled as an optional IM channel. A Discord bot receives commands such as !opengui devices or !opengui do ..., then the backend dispatches the task to a standby Android phone and posts progress back to the same channel.

This is not required for local use. If DISCORD_BOT_TOKEN is empty, the backend starts normally and skips Discord.

Full setup guide: docs/DISCORD.md.

The System

flowchart LR
    U["User or IM command"] --> BS["Bootstrap Skill / API / IM entry"]
    BS --> SP["Plan Supervisor"]

    SP --> EX["Executor Graph"]
    EX --> AC["Android Client"]
    AC --> GX["AccessibilityService + screenshots + actions"]
    EX --> RV["Execution review and retry"]
    RV --> SP

    SP --> SM["Summarizer"]
    SM --> SR["Structured Results"]

    RD["Feishu / Telegram / Discord / REST API"] --> ST["Standby Gateway"]
    ST --> AC

    SP --> MR["Model Routing"]
    MR --> MA["Claude / GPT / Gemini / Kimi / MiniMax / compatible"]
    EX --> MR

Core Runtime Pieces

Backend graph: server/apps/backend/src/modules/graph-agent/graph/
Task APIs: server/apps/backend/src/modules/task/task.controller.ts
Standby dispatch: server/apps/backend/src/common/ws/standby.gateway.ts
IM channel dispatch: server/apps/backend/src/modules/im-channel/
Android standby connection: client/core_network/src/main/java/com/coremate/opengui/network/websocket/StandbySocketManager.kt
Android execution path: client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.kt

Documentation

Community / Support

The most useful project feedback is:

open issues for bugs and feature requests
share real use cases and deployment feedback
contribute docs, integrations, and fixes

License

OpenGUI is source-available under the Business Source License 1.1 (BUSL-1.1).

You may copy, modify, distribute, and use the source for non-production purposes. Production use, commercial use, hosted services, and integration into commercial products require a separate commercial license from Core-Mate.

For this version:

Change Date: 2030-04-29
Change License: Apache License, Version 2.0

This is public source, but it is not OSI-approved open source until the Change Date.

See LICENSE.

File tree

1679

Core-Mate/OpenGUIView on GitHub

.github/
  ISSUE_TEMPLATE/
    bug_report.yml
    config.yml
    feature_request.yml
    setup_help.yml
  pull_request_template.md
.gitignore
AGENTS.md
CHANGELOG.md
CLAUDE.md
client/
  .gitignore
  app/
    .gitignore
    build.gradle.kts
    proguard-rules.pro
    src/
      androidTest/
        java/
          com/
            coremate/
              opengui/
                ExampleInstrumentedTest.kt
      main/
        AndroidManifest.xml
        assets/
          coze_oauth_config.json
          data.json
          images/
            img_0.png
            img_1.png
            img_10.png
            img_100.png
            img_101.png
            img_102.png
            img_103.png
            img_104.png
            img_105.png
            img_106.png
            img_107.png
            img_108.png
            img_109.png
            img_11.png
            img_110.png
            img_111.png
            img_112.png
            img_113.png
            img_114.png
            img_115.png
            img_116.png
            img_117.png
            img_118.png
            img_119.png
            img_12.png
            img_120.png
            img_121.png
            img_122.png
            img_123.png
            img_124.png
            img_125.png
            img_126.png
            img_127.png
            img_128.png
            img_129.png
            img_13.png
            img_130.png
            img_131.png
            img_132.png
            img_133.png
            img_134.png
            img_135.png
            img_136.png
            img_137.png
            img_138.png
            img_139.png
            img_14.png
            img_140.png
            img_141.png
            img_142.png
            img_143.png
            img_144.png
            img_145.png
            img_146.png
            img_147.png
            img_148.png
            img_149.png
            img_15.png
            img_150.png
            img_151.png
            img_152.png
            img_153.png
            img_154.png
            img_155.png
            img_156.png
            img_157.png
            img_158.png
            img_159.png
            img_16.png
            img_160.png
            img_161.png
            img_162.png
            img_163.png
            img_164.png
            img_165.png
            img_166.png
            img_167.png
            img_168.png
            img_169.png
            img_17.png
            img_170.png
            img_171.png
            img_172.png
            img_173.png
            img_174.png
            img_175.png
            img_176.png
            img_177.png
            img_178.png
            img_179.png
            img_18.png
            img_180.png
            img_19.png
            img_2.png
            img_20.png
            img_21.png
            img_22.png
            img_23.png
            img_24.png
            img_25.png
            img_26.png
            img_27.png
            img_28.png
            img_29.png
            img_3.png
            img_30.png
            img_31.png
            img_32.png
            img_33.png
            img_34.png
            img_35.png
            img_36.png
            img_37.png
            img_38.png
            img_39.png
            img_4.png
            img_40.png
            img_41.png
            img_42.png
            img_43.png
            img_44.png
            img_45.png
            img_46.png
            img_47.png
            img_48.png
            img_49.png
            img_5.png
            img_50.png
            img_51.png
            img_52.png
            img_53.png
            img_54.png
            img_55.png
            img_56.png
            img_57.png
            img_58.png
            img_59.png
            img_6.png
            img_60.png
            img_61.png
            img_62.png
            img_63.png
            img_64.png
            img_65.png
            img_66.png
            img_67.png
            img_68.png
            img_69.png
            img_7.png
            img_70.png
            img_71.png
            img_72.png
            img_73.png
            img_74.png
            img_75.png
            img_76.png
            img_77.png
            img_78.png
            img_79.png
            img_8.png
            img_80.png
            img_81.png
            img_82.png
            img_83.png
            img_84.png
            img_85.png
            img_86.png
            img_87.png
            img_88.png
            img_89.png
            img_9.png
            img_90.png
            img_91.png
            img_92.png
            img_93.png
            img_94.png
            img_95.png
            img_96.png
            img_97.png
            img_98.png
            img_99.png
          mock_chat_group.jpg
          mock_moment.jpg
          voice_data.json
          voice_images/
            img_0.png
            img_1.png
            img_10.png
            img_100.png
            img_101.png
            img_102.png
            img_103.png
            img_104.png
            img_105.png
            img_106.png
            img_107.png
            img_108.png
            img_109.png
            img_11.png
            img_110.png
            img_111.png
            img_112.png
            img_113.png
            img_114.png
            img_115.png
            img_116.png
            img_117.png
            img_118.png
            img_119.png
            img_12.png
            img_120.png
            img_121.png
            img_122.png
            img_123.png
            img_124.png
            img_125.png
            img_126.png
            img_127.png
            img_128.png
            img_129.png
            img_13.png
            img_130.png
            img_131.png
            img_132.png
            img_133.png
            img_134.png
            img_135.png
            img_136.png
            img_137.png
            img_138.png
            img_139.png
            img_14.png
            img_140.png
            img_141.png
            img_142.png
            img_143.png
            img_15.png
            img_16.png
            img_17.png
            img_18.png
            img_19.png
            img_2.png
            img_20.png
            img_21.png
            img_22.png
            img_23.png
            img_24.png
            img_25.png
            img_26.png
            img_27.png
            img_28.png
            img_29.png
            img_3.png
            img_30.png
            img_31.png
            img_32.png
            img_33.png
            img_34.png
            img_35.png
            img_36.png
            img_37.png
            img_38.png
            img_39.png
            img_4.png
            img_40.png
            img_41.png
            img_42.png
            img_43.png
            img_44.png
            img_45.png
            img_46.png
            img_47.png
            img_48.png
            img_49.png
            img_5.png
            img_50.png
            img_51.png
            img_52.png
            img_53.png
            img_54.png
            img_55.png
            img_56.png
            img_57.png
            img_58.png
            img_59.png
            img_6.png
            img_60.png
            img_61.png
            img_62.png
            img_63.png
            img_64.png
            img_65.png
            img_66.png
            img_67.png
            img_68.png
            img_69.png
            img_7.png
            img_70.png
            img_71.png
            img_72.png
            img_73.png
            img_74.png
            img_75.png
            img_76.png
            img_77.png
            img_78.png
            img_79.png
            img_8.png
            img_80.png
            img_81.png
            img_82.png
            img_83.png
            img_84.png
            img_85.png
            img_86.png
            img_87.png
            img_88.png
            img_89.png
            img_9.png
            img_90.png
            img_91.png
            img_92.png
            img_93.png
            img_94.png
            img_95.png
            img_96.png
            img_97.png
            img_98.png
            img_99.png
        java/
          com/
            coremate/
              opengui/
                login/
                  login/
                    adapter/
                      LoginPagerAdapter.kt
                    Controller.kt
                    fragment/
                      GuidePageFragment.kt
                      LoginPageFragment.kt
                      PermissionPageFragment.kt
                      PhoneNumberFragment.kt
                      VerificationCodeFragment.kt
                    IntroGuideActivity.kt
                    LoginActivity.kt
                    UserGuideActivity.kt
                    view/
                      InputCodeLayout.java
                  PermissionDialogItem.kt
                  PromotorApplication.kt
                  ProtocolPopupDialogFragment.kt
                  SoundWavesView.kt
                  SplashActivity.kt
        res/
          color/
            bottom_nav_item_color.xml
            user_guide_continue_text.xml
          drawable/
          drawable-xxxhdpi/
            bg_dec.jpg
            general_bg_dec.jpg
            ic_ai_text.png
            ic_nav_selected.png
            bg_aurora_login.xml
            bg_btn_primary_shadow.xml
            bg_dialog_progress.xml
            bg_edit_error.xml
            bg_edit_focus.xml
            bg_edit_un_focus.xml
... 1600 more

Showing a partial view of a very large repo.

FAQ

opengui is a Claude Code plugin with 2 hand-picked skills for automation work, indexed on Flowy. Install it with the command on its page. It includes open-gui-bootstrap, open-gui-remote-control. Its skills do not fire on their own yet. Request auto-invocation to have Flowy route them as you prompt. Free and open source.

What is opengui?: OpenGUI is an Android GUI agent framework for phone-use AI that can see, plan, and operate real mobile apps through the GUI.
How do I install opengui?: Run these in Claude Code: npx -y skills add Core-Mate/OpenGUI --agent claude-code. Then prompt normally.
Does opengui auto-invoke its skills?: Not yet. It is indexed on Flowy as a plain plugin. Request auto-invocation on its page and Flowy will route its skills for you as you prompt.
Is opengui free?: Yes. Flowy is free and open source, with nothing gated. You can read every skill in full before you install.