Does sutando auto-invoke its skills?

Not yet. It is indexed on Flowy as a plain plugin. Request auto-invocation on its page and Flowy will route its skills for you as you prompt.

Yes. Flowy is free and open source, with nothing gated. You can read every skill in full before you install.

Automation

Plugin

sutando

My AI Stand — Realtime by Day, Rewriting Itself by Night. Summon my AI superpower. Voice, vision, screen, meetings, calls when I'm engaged. Learns my patterns, ships its own code when I'm not. Runs across my Macs, interacts with people & their Stands.

FLOW.md

sonichi@sonichi

GitHub

Plugin Signal89/100Strong

maintained100

capable87

documented100

adopted55

Install

$ npx -y skills add sonichi/sutando --agent claude-code

Run the curl in your terminal, the rest in Claude Code.

Stats

TypeScript JavaScript Node Bash Swift

+7 tags

Git GitHub Anthropic OpenAI Gemini Slack Obsidian

365

Stars

Forks

Active

Maintenance

Python

Language

MIT

License

48m ago

Last commit

4mo ago

Created

Repo: sonichi/sutando

In this plugin

57 skills 7 hooks

See everything inside

What's inside

Skills

/agent-registryLocal Agent Registry — a standalone, dependency-free service that tracks running Claude Code (and other) agent instances. Agents self-register on startup and heartbeat while alive; the Electron overlay and Sutando dashboard read the live list. Use when you need to know which /agent-room-ops**One skill, multiple tools.** Everything an agent does in a room beyond its task inbox lives here as a tool, so the parity capabilities are self-evidently *one collection* (not N scattered skills). Each tool is a thin **gateway-only** client verb sharing `_gateway.py`; the /audio-transcribeTranscribes audio files and voice notes to text via Gemini 2.5-flash. Integrates with Slack, Discord, and Telegram bridges so voice clips surface as readable text in tasks./bot2bot-postPost a coordination message from this bot to the shared bot2bot channel — @-mentioning a specific peer via --to, auto-mentioning only in single-peer fleets, never guessing./call-diagnosticsAnalyze phone call observability data, detect problems, track them across calls, and recommend systematic repairs./claude-codexBash wrapper around the local Codex CLI for non-interactive runs from inside Sutando (bridges, cron, scripts). For interactive code review or task hand-off from this Claude Code session, prefer the official `/codex:*` plugin commands; this skill is the file-bridge-compatible

Show all 57 skills

/claude-geminiUse the local Gemini CLI from Claude Code with the user's existing Gemini authentication or API configuration. Use for large-context repo scans, multimodal analysis, second-opinion planning, or structured Gemini runs in the current workspace./claude-routerChoose between the local Codex CLI and Gemini CLI from Claude Code. Use for automatic model selection when the user wants the best local delegate for code review, repo-wide analysis, planning, or implementation./context-dropTiny Swift CLI that reads the focused app's text selection via the macOS Accessibility API. Sutando.app's "drop context" action (hotkey configurable via `state/hotkeys.json`) shells out to this binary to capture what you have highlighted./context-reconstructRe-anchor on the durable record (current-track, live owner thread, pending-questions, relay, build_log) before acting on anything that depends on earlier context. Read, do not recall./db-browserInstall DB Browser for SQLite (if not already installed) and open a .sqlite file in it. macOS only./deal-finderScan configured sources (Craigslist now; eBay + Facebook Marketplace planned) for used-item listings matching the owner's criteria. Currently configured for a Mac mini search (M2+, 16GB+, 512GB+, ≤$500, near 94566). Notify owner via SMS + Telegram on a match./doc-ingestExtract readable text from document files — PDF, XLSX/CSV/TSV, DOCX, PPTX, and plain-text formats — so any task that arrives with an attachment can actually consume it./electron-overlay-dimmingReusable pattern for focus-based auto-dimming of Electron overlay windows — when the app loses focus, all overlay windows fade to a low opacity; when an overlay regains focus, they return to their configured opacity. Use when building always-on-top Electron overlays that should /email-findLocate a specific email when the obvious searches fail. Use when the user is confident an email exists but a targeted query returned nothing./gemini-ttsRender text to mp3 via Google Gemini Flash TTS. Free-tier eligible (1500 req/day). Use for video narration, demo voiceovers, audio notes. Parallels openai-tts; default for make-viral-video./image-generationGenerate and edit images using Gemini Flash Image, and generate videos using Veo. Supports text-to-image, image editing, text-to-video, and image-to-video./macos-toolsmacOS native integrations: screen capture, calendar, reminders, contacts, email (Mail.app), Spotlight search. Use when the user asks about their screen, schedule, to-do list, contacts, or wants to send email on macOS./macos-useGUI control for macOS apps via mediar-ai's mcp-server-macos-use. Click, type, scroll, key-press, open apps — driven by accessibility tree, works in non-interactive Claude Code mode. Use this for any Sutando task that needs to drive another macOS application (Safari, Zoom, Mail,/make-viral-videoBuild a short news-explainer video tuned for shareability. One striking moment per video; real fetched assets; self-heal validation; pluggable TTS (Gemini-free default, OpenAI fallback)./media-transcriptPull a video's transcript (captions/subtitles) as clean text — the doc-ingest sibling for video. Any task that references a YouTube (or other yt-dlp-supported) URL can consume the spoken content as text instead of being blind to it./meeting-prepAuto-prepare for upcoming meetings: attendee info, recent email threads, talking points, and agenda. Runs 30 min before each meeting or on demand./meeting-schedulerSchedule a small meeting end-to-end: resolve attendee emails, check the owner's calendar for the slot, dedup-check, then create + email the Google Calendar invite. The mechanical core only — cross-person availability negotiation stays interactive./morning-briefingGenerate a daily morning briefing: email, calendar, Discord, and news — delivered via voice or Discord DM./ms365-connectConnect to Microsoft 365 (OneDrive, Outlook mail, Calendar, Teams) via Microsoft Graph using the open-source python-o365 (O365) library. Use when the user wants to list/download OneDrive files, read or send Outlook email, list calendar events, or post a message to a Teams /observe/observe — natural-language entry to the Policy Interaction Layer: compile an NL request into a structured observation/subscription policy, confirm via effect-card (or standing approval), then subscribe on the events plane./obsidian-vaultVoice-inline capture into a Sutando-owned Obsidian vault. The voice agent calls `add_to_vault(kind, body, title?)` directly — no core round-trip, no Obsidian plugin required. Filesystem-direct: Obsidian's watcher picks up the change instantly when the vault is open./open-sutando-refResolve a fuzzy GitHub reference (issue number, PR number, or text description) to a URL and optionally open it in the browser./openai-ttsRender text to mp3 via OpenAI's tts-1-hd. Use for video narration, demo voiceovers, audio notes./overlay-appsFramework + minimal example for Sutando desktop overlay applications — always-on-top, frameless Electron windows that float over the desktop, controllable from the Sutando web UI's /overlays manager. Ships one example overlay (System Resources). Add new overlays by registering /phone-conversationMake conversational phone calls and join Zoom meetings via Twilio + Gemini. Multi-turn AI conversations on the phone on behalf of the user./proactive-loopStart Sutando's autonomous proactive loop. Monitors tasks, runs health checks, and builds missing capabilities on a recurring schedule./quota-trackerTrack Claude Code quota usage via Anthropic API rate limit headers. Shows 5h and 7d utilization, reset times, and quota status. Works with both subscription and API key auth./regression-searchSearch phone-call history for when a feature regressed (find-regression.py) and drill into a single call to see what went wrong (diagnose-call.py). Skips reading 100+ transcripts by hand./relayWrite a handoff/continuity note for the NEXT Sutando session. Captures what was just in flight, what to check first, what might go wrong, and implicit context the structured snapshot doesn't carry. Read first by /catchup-after-startup./releasePrepare, verify, and publish Sutando engine releases. Use when asked to prepare a release proposal, determine a version bump, audit release documentation, validate release gates, create a release PR, tag a confirmed release, or publish GitHub release notes. Preparation is safe /report-feedbackFile a bug report, feature request, or feedback about Sutando to the team from any surface (chat, Discord, Telegram, or a voice-delegated task). Reuses the cloud /api/feedback API and auto-attaches diagnostic context. Use when the user says "report a bug", "something's broken,/schedule-cronsRe-create all session cron jobs for Sutando. Run this on startup or after a session restart./screen-companionSutando watches your screen and helps in real time, with the interaction pattern pre-configured per use case. Reads papers with you, pair-debugs, reviews PRs — without you having to narrate intent every session./screen-recordStart and stop screen recording using ffmpeg./self-diagnoseSutando introspection — read logs + git + memory + build log for a chosen time window and produce a concise narrative of what the agent has been doing, what's broken, and what to prioritize next./self-upgradeSafely upgrade this Sutando checkout to the latest upstream code **without bricking the running core session** — the "success path" distilled from a real 2026-07-20 upgrade that would otherwise hang (and did, the first time)./session-recapReconstruct what happened in a past core session — from a high-level summary down to verbatim owner quotes — by reading the raw session transcripts (complete, crash-proof, unbiased), not the curated relay/handoff notes./skill-usage-reportEngine half of the skill-usage telemetry loop (owner spec 2026-07-19: *"show when the skill is last used, and how many times… recommend disable, or user would disable themself"*). Cloud half: AU#93 (`skill_usage` rollup + `POST /api/skills/usage` + last-used in the assignments /startupSingle entry point for fresh-session bootstrap. Runs optional task-orphan recovery, cron registration, and watcher start in a fixed order. Replaces the current `claude -- '/schedule-crons'` invocation pattern as the canonical CLI startup target./submit-use-caseOSS-facing channel to share a Sutando use case with the community. Opens a labeled GitHub issue on sonichi/sutando and (CLA-gated) a PR adding docs/community-use-cases/<slug>.md. Validates outcome-framed titles with the same checker as add-use-case. Default: both; flags /subscription-scannerScans Gmail for active paid subscriptions and tracks them over time. Surfaces additions / cancellations between scans./sutando-migrateM1 Part 2 workspace migration — guided walkthrough that scans legacy state across sources A (repo-root), B (~/.sutando/workspace/), C ($SUTANDO_WORKSPACE env-override), surfaces collisions, gets owner greenlight, commits, verifies, and (if sutando-plus + sync configured)/task-orphan-checkResolve orphan tasks left in `<workspace>/tasks/` from a previous session that crashed mid-execution. Classifies each live task as done / fresh / stale by cross-referencing per-side-effect markers, then archives or recovers as appropriate. Runs once on startup; safe to re-invoke./task-progressSends mid-task progress updates to the channel a task came from (Slack, Discord, or Telegram)./task-workstream-groupingInfer stable semantic workstreams for ungrouped Sutando task history and apply the validated assignments to the durable workstream sidecar. Use for internal task-workstream-grouping maintenance tasks, initial history backfills, and later batches of tasks that need cross-channel /task-workstream-sessionsIsolate already-assigned owner tasks into durable provider sessions keyed by inferred workstream. This runtime skill is adapter-invoked; ungrouped tasks keep the selected core's legacy session./trusted-capabilitiesDiscover, evaluate, safely install, and update reusable skills from a curated set of trusted open-source repositories./voice-agent-test-harnessDrive a fixed suite of spoken tests against a voice agent ("subject") from a co-located machine ("prober"), measure response latency / clarity / accuracy, diff against baseline, and report to the owner over Telegram./whatsappSend WhatsApp messages, list chats, and search history via wacli (local CLI backed by a synced store at ~/.wacli). Use after the user has run `wacli auth` to pair the device./x-twitterPost tweets, search, read mentions, and check engagement on X (Twitter) via API v2./zoomZoom meeting control for Sutando's voice and phone agents — contributes the summon (join + screen share), dismiss (leave meeting), and join_zoom (join with computer audio, no share) inline tools.

Hooks

activity-emitter.pyHook script.context-source-guard.pyHook script.gmail-write-guard.pyHook script.human-action-bridge.pyHook script.result-file-marker-guard.pyHook script.skill-usage-telemetry.pyHook script.

Show all 7 hooks

skip-ask-user-question.pyHook script.

Badges

README

Read on GitHub

Sutando

My AI Stand — Realtime by Day, Rewriting Itself by Night. Summon my AI superpower.

Voice, vision, screen, meetings, calls when I'm engaged. Learns my patterns, ships its own code when I'm not. Runs across my Macs, interacts with people & their Stands.

It belongs entirely to you.

🛠 Open source: this repo — clone, build, run locally on your own Mac. 🍎 Native app preview: sutando.ai — packaged Mac app, request access.

No pay-per-token core API key required. Sutando runs through the Claude Code or Codex CLI session you select, with optional service credentials only for the capabilities you enable.

Named after Stands from JoJo's Bizarre Adventure — a personal spirit that fights on your behalf. Like a Stand, Sutando starts unnamed. As it learns your style and earns real capabilities, it names itself and generates its own avatar — your Stand, unique to you.

https://github.com/user-attachments/assets/a86ec34e-3b26-4011-824c-d2d124753c25

24 tool calls. 6 tasks. 7 minutes. All by voice from a phone. Demo by @liususan091219. Watch on YouTube (full quality) →

⭐ If Sutando is useful to you, star this repo — it's how other people find it.


🤖 The AI agent that presented live	📺 ep.007 — 50 days. 600+ PRs. #1 trending	📺 Vision talk at UC Berkeley

Sutando in action

@sutando-ai channel · Sutando WIRE playlist

What can you do with it?

Talk while you work. You're looking at a doc. You say "make this paragraph shorter." Sutando sees your screen, rewrites the paragraph, and replaces the original text directly.

Join meetings for you. "Join my 2pm call." It reads your calendar and joins — Zoom via the desktop app, Google Meet via the browser — with computer audio. It can also dial in by phone when you ask. It takes screenshots to identify participants, does live research when someone asks a question, and writes you a summary when the call ends. Meeting access is gated — it messages you on Telegram asking for approval before enabling task delegation.

Make calls for you. "Call her and leave a message." Sutando looks up the contact, dials the number, has the conversation, and reports back — while you keep working. It can even make concurrent calls while in a meeting.

Work from your phone. Call Sutando and say "summon." It opens Zoom with screen sharing — join from your phone to see its screen in real time. "What's on my screen?" — it takes a screenshot and tells you. "Fix the typo in that file" — done. You scroll, switch apps, navigate — all by voice while walking around.

Get better on its own. When you're not giving it tasks, Sutando runs an autonomous build loop — it monitors its own health, detects patterns in how you work, discovers new skills, and builds missing capabilities. Most of Sutando's code was written this way. It learns from your corrections and adapts over time.

Remember everything — and act on it. You have an idea while walking. Say it out loud. Sutando captures it, tags it, and saves it as a searchable note. If there's something actionable, it starts working on it right away or queues it for the next free cycle.

Reach you anywhere. Voice, Zoom, Google Meet, Telegram, Discord (text + voice channel), web, phone, or email — same agent, same memory, any channel.

Scale across machines. Plug in a second Mac and Sutando sets it up — the original agent opens a Discord channel, sends setup commands, and migrates services. The new machine handles phone calls 24/7 while your laptop stays portable. No migration scripts needed — the two agents coordinate the handoff themselves.

Status: Alpha

This is an early-stage project. Honest status:

	Count	Details
Verified working	30	Voice, screen capture, notes, calendar, reminders, contacts, browser, phone calls, meeting dial-in, task delegation, pattern detection, health check, dashboard, Telegram, Discord, multi-machine migration, onboarding tutorial, and more
Needs external setup	3	Twilio (phone), Telegram bot, Discord bot

We're looking for contributors to help test and harden these capabilities. If you try something and it breaks, open an issue.

How it works

See Sutando architecture boundaries for the normative definitions of core, adapters, apps, skills, tooling, and workspace state.

    You ──voice (browser)──► Voice agent ─────────┐
     │                       (Gemini Live,        │
     │                        WS on :9900)        ├──► inline tools (instant,
     │                                            │    in-process: describe_screen,
     ├──phone (Twilio)─────► Conversation server ─┤    get_current_time, hang_up,
     │                       (Gemini Live,        │    dtmf, ...)
     │                        WS on :3100)        │
     │                                            └──┐
     │                                               │   file bridge       .──────▶────────.
     ├──telegram──────────► Telegram bridge ─────────┼── tasks/ ─────────► |               |
     │                                               │                    |   Core        |
     │                                               │                    |   agent ↻     |
     └──discord───────────► Discord bridge ──────────┘                    |               |
                                                                           `──────◀────────'
                                                                                  │
                                                                                  ▼
                                                                          uses anything:
                                                                          email, calendar,
                                                                          browser, files,
                                                                          phone, reminders...
                                    ◄── results/ ◄────────────────────────────────┘
                                (spoken via voice/phone,
                                 text via Telegram/Discord)

    ↻ = a cron job fires the `/proactive-loop` skill every 15 minutes
        (`*/15 * * * *` in the per-host `crons.json`). The skill
        runs as a 10-minute pass that keeps a persistent watcher on
        `tasks/` via Claude Code's `Monitor` tool — pending tasks are
        processed the moment they arrive, not just on the cron tick.
        Each pass also runs health checks and picks the next build-log
        item autonomously.

Four processes work together:

Voice agent (Gemini Live, WebSocket on :9900) — listens and talks in real time for browser voice.
Web client (com.sutando.web-client.plist, HTTP on :8080) — separate launchd service that serves the browser UI. The browser then connects directly to the voice agent's WebSocket on :9900 — the web client is not in the WebSocket data path.
Conversation server (Gemini Live, Twilio WebSocket on :3100) — same role as the voice agent for inbound and outbound phone calls.
Core agent (Claude Code or Codex CLI) — executes tasks with full system access. The persistent CLI session provides an interactive terminal and runs Sutando's task watcher and scheduled work.

Voice agent and conversation server handle conversation-scope actions with inline tools — in-process calls that round-trip instantly (describe the screen, hang up, send DTMF, read the clipboard/current time, capture a screenshot). For anything outside that scope they write to tasks/; core reads them, executes, and writes to results/, which each channel speaks or messages back. Telegram and Discord bridges only use the tasks/ path.

Quick start

Prerequisites — bash src/startup.sh checks that these are installed and refuses to boot otherwise:

Claude Code or Codex CLI — whichever you select
Node.js (brew install node)
Python 3 (brew install python3)
fswatch (brew install fswatch) — auto-installs via Homebrew on first start

Also required — sign in to the selected agent CLI. Startup checks the configured Claude or Codex home before launching background services and fails with the matching login remedy. SUTANDO_SKIP_AUTH_PREFLIGHT=1 bypasses this once for recovery; the runtime launcher still checks again before replacing the core session.

Recommended, but not checked at boot — macOS 15+ and Node.js 22+.

bash src/verify-setup.sh covers this second list — it checks the Node version and whether your CLI is actually authenticated. Run it if startup succeeds but the core doesn't.

Optional — each unlocks one feature and degrades alone:

Gemini API key — voice (text/core paths work without it)
pip3 install discord.py / slack_bolt — Discord / Slack bridges (Telegram needs no package)
ffmpeg (brew install ffmpeg) — subtitle-burn, video-concat, recording handoff
tmux (brew install tmux) — Sutando.app watcher auto-restart; the core starts without it
git — vault sync, self-upgrade, commit provenance
Twilio account + ngrok — phone calls and SMS

Full list with the line enforcing each, plus what to vendor when embedding Sutando in another application: External runtime dependencies.

# Clone
git clone https://github.com/sonichi/sutando.git
cd sutando

# Configure optional integrations (skip for text/core-only use)
cp .env.example .env
# Add GEMINI_API_KEY only if you want voice

# Start everything
bash src/startup.sh

This starts the headless core services (voice agent, phone conversation server, web client, dashboard, and API). Open http://localhost:8080 when you want the browser UI; startup never opens a browser or launches a macOS app for you. The autonomous loop starts automatically.

Why Sutando runs with elevated permissions. Autonomous voice-driven work means startup.sh launches the selected core CLI with unattended approvals and full local access — permission prompts would otherwise break the voice-in / answer-out flow. In exchange:

It's local. Sutando runs entirely on your Mac. No remote control plane, no third party with write access.

You control the audience. 3-tier access gating means owner / verified / unverified callers get different capability bands on phone, Discord, and Telegram. Set VERIFIED_CALLERS in .env before going live.

Actions are auditable. Every task lands in tasks/ + results/, and service activity is written to logs/*.log. Use the Core CLI terminal while it works to watch in real time.

Hooks are your brake pedal. git-rules-guard.sh (see $CLAUDE_CONFIG_DIR/hooks) pops a Discord approval DM for any public write (push / PR / issue comment) regardless of transport. Reject with 👎 to block.

Keep the Core CLI terminal reachable — quota exhaustion or an unrecognized CLI prompt can leave the core agent waiting for you to respond. See Codex core setup to select or roll back the Codex runtime.

Why macOS 15+? The setup scripts assume the Sequoia System Settings layout for granting TCC permissions (Screen Recording, Accessibility, Input Monitoring). Earlier macOS versions may work for the headless parts (proactive loop, Discord/Telegram bridges) but aren't tested.

macOS permissions — on first run, macOS will ask you to grant Screen Recording, Accessibility, and Microphone access. See Security for what each permission is used for.

Try saying:

"What's on my screen?" — takes a screenshot and describes it
"Summon my computer to zoom" — opens Zoom with screen sharing, join from your phone
"Join my next meeting" — checks your calendar and joins
"Take a note: my first idea" — saves a searchable note
"Tutorial" — walks you through all capabilities step by step

Verify your setup (optional):

bash src/verify-setup.sh

Troubleshooting:

Browser shows blank page? Services may still be starting — wait 5 seconds and refresh
Microphone not working? Chrome will ask for permission on first connect — click Allow
Voice agent not responding? Check logs/voice-agent.log for errors. Common causes:
- GEMINI_API_KEY not set or invalid in .env — get one at ai.google.dev
- Port 9900 already in use — run lsof -i :9900 to check
npm install failed? Make sure Node.js 22+ is installed: node --version
Gemini 429 errors? Your shell may have a stale GEMINI_API_KEY overriding .env — run unset GEMINI_API_KEY then restart
Screen recording produces 0-second files? screencapture -v needs a TTY. Sutando uses ffmpeg instead — make sure it's installed: brew install ffmpeg
Something broke? Run bash src/restart.sh — this kills all services and restarts fresh
Sutando acting confused, contradicting itself, or giving stale answers after a long session? Restart the selected core CLI session to reset its context.
Still stuck? Join the official Discord — real humans and community-run agents answer support questions there.
Phone call answers with "We are sorry, an error has occurred"? The conversation server (skills/phone-conversation/scripts/conversation-server.ts, port 3100) isn't running. Run bash src/startup.sh or bash src/restart.sh to relaunch all services.

Shutting down:

bash src/restart.sh    # stops all services (voice agent, web client, API, bridges, etc.)
pkill -x Sutando # stop the menu bar app

Exiting startup.sh alone does NOT stop background services. Always use restart.sh (or kill-all.sh if available) to cleanly shut everything down.

Uninstalling:

Stop all services: bash src/restart.sh && pkill -x Sutando
Remove the repo: rm -rf ~/Desktop/sutando (or wherever you cloned it)
Remove config: rm -rf $CLAUDE_CONFIG_DIR/projects/*sutando*
Remove npm packages (optional): the repo uses local node_modules/ — deleted with the repo
Remove any tools you installed during setup (e.g. imsg, wacli) via the package manager you used to install them.
If you installed the OS-supervised health checks: bash src/install-health-check-launchd.sh --uninstall (idempotent — no-op if not installed).

Optional integrations

These unlock more capabilities. Add to .env when ready:

Integration	What it unlocks	Setup
Gmail	Read/send/search email from voice	`gws auth setup --login` (OAuth, no app password)
Twilio + ngrok	Phone calls, SMS, meeting dial-in, task delegation via phone	twilio.com + `brew install ngrok` (see Running costs)
Telegram	Message Sutando from your phone. First DM auto-enrolls you as owner (trust-on-first-use). Subsequent senders need to be added: edit `$CLAUDE_CONFIG_DIR/channels/telegram/access.json` → `allowFrom` list.	Create bot via @BotFather, then `/telegram:configure <token>`
Discord	Message Sutando from Discord (DM + channel @mentions)	Developer portal, then `/discord:configure <token>`
Claude for Chrome	Browser automation — navigate, read pages, fill forms, interact with web apps	Install extension, log in with the same account as Claude Code
Sutando app (menu bar)	Optional global hotkeys (see Keyboard shortcuts)	Build and launch separately; core startup stays headless
OS-supervised health checks	Detect stuck loops, dead watchers, and queue pileups even when core is unresponsive — macOS notifies you when Sutando is broken	`bash src/install-health-check-launchd.sh` (idempotent; uninstall with `--uninstall`)
Multi-machine workspace sync	Run the same agent identity across Mac mini + MacBook + Mac Studio etc.; memory + notes + state stay consistent via a private git repo you own	Create a private vault repo, set `vault.remote_url` in `sutando.config.local.json`, run `bash scripts/sync-workspace.sh --init` once + cron it. See docs/workspace-sync.md. The legacy `sync-memory.sh` flow is deprecated in v0.3.0 and removed in v0.4.0.

Running costs

One table, organized by capability. Core access comes from the Claude Code or Codex CLI account you select; the remaining services are optional and mostly free-tier-sufficient.

Capability	When you need it	Service required	Cost
Basic (core agent + screen / notes / calendar / reminders / contacts / browser / iMessage)	Always — this is Sutando's baseline	Claude Code or Codex CLI access + macOS	Depends on the selected CLI account. No Gemini key or Google OAuth is required for core operation.
Voice agent (real-time conversation in browser or on phone)	If you want to talk to Sutando	Gemini voice API	Free tier covers normal use (~15 req/min). Heavy use: Gemini paid ~$0.30–$1.30/hr.
Telegram / Discord / WhatsApp (message Sutando from any of these)	If you want non-voice chat from your phone or desktop	Telegram BotFather, Discord developer portal, `wacli` (bundled)	All free for personal use.
Phone calls / summon (remote control)	If you want Sutando to make inbound/outbound calls, or to share its computer screen via Zoom/Google Meet and be controlled by voice from your phone	Twilio phone number + ngrok webhook	Twilio ~$1/mon number + ~$0.0085/min inbound + ~$0.015/min outbound + Media Streams ~$0.004/min. ngrok and Zoom free tiers both work for the summon flow.
Agent joining meetings via dial-in (PSTN join into Zoom / Google Meet)	If you want the phone agent to dial into a meeting as a participant	Zoom Pro OR Google Workspace Business on the host side (the meeting organizer's account needs toll dial-in enabled)	Zoom Pro ~$15/mon, Google Workspace Business Starter ~$7/mon. Sutando's side is already covered by the Phone row above.

Minimal-cost path: use your selected core CLI account for text, browser, and messaging. Add Gemini for voice, Google OAuth for Google services, and Twilio for phone or meeting dial-in only when needed.

What's inside

Capability	Script	Status
Voice conversation	`voice-agent.ts`	Verified
Task delegation (voice → core)	`task-bridge.ts` + `watch-tasks-stream.sh` + `tasks/` dir	Verified
Screen capture + analysis	`macos-tools` skill	Verified
Notes / second brain	`notes/` directory (YAML-frontmatter markdown)	Verified
Context drop + shortcuts	`src/Sutando/` menu bar app	Verified
Gmail read/send/search	`gws-gmail` skill	Verified
Calendar reading	`google-calendar` skill	Verified
Reminders management	`macos-tools` skill	Verified
Contacts lookup	`macos-tools` skill	Verified
Browser automation	`browser.mjs` + MCP tools	Verified
Conversational phone calls	`phone-conversation/`	Verified (needs Twilio + ngrok)
Phone → task delegation	`phone-conversation/`	Verified (needs Twilio + VERIFIED_CALLERS)
Join Zoom (computer audio)	`inline-tools.ts`	Verified
Join Google Meet (browser audio)	`inline-tools.ts`	Verified
Meeting dial-in (Meet + Zoom)	`phone-conversation/`	Verified (needs Twilio + ngrok)
Meeting approval via Telegram	`phone-conversation/`	Verified (needs Twilio + Telegram)
Inbound call handling	`phone-conversation/`	Verified (needs Twilio)
Telegram messaging	`telegram-bridge.py`	Verified (text + photos + files + voice)
Discord messaging	`discord-bridge.py`	Verified (DMs + channel @mentions + files)
Cross-device task submission	`agent-api.py`	Verified
Health monitoring	`health-check.py`	Verified
Pattern detection + user modeling	Core memory files + selected CLI	Verified
System dashboard	`dashboard.py`	Verified
Info-radar (arXiv / GitHub / HN / news monitoring)	`info-radar` skill + daily digest	Verified
Menu-bar avatar states (idle/listening/speaking/working)	`src/Sutando/main.swift` + `/sse-status`	Verified

Services

When running, Sutando exposes these local ports:

Port	What
8080	Voice web client — talk to Sutando here
7844	Dashboard — status, activity, and capability matrix
7843	Agent API — submit tasks from any device
9900	Voice agent WebSocket
7845	Screen capture server
3100	Phone conversation server (Twilio webhook target)
4040	ngrok admin UI (when ngrok is running)

Keyboard shortcuts

The optional Sutando menu bar app (src/Sutando/) provides global keyboard shortcuts. It is separate from the headless core and is never built or launched by startup.sh. All shortcuts are configurable — the bindings below are the shipped defaults, published at runtime to <workspace>/state/hotkeys.json (the source of truth); override any of them per-machine in ~/.config/sutando/hotkeys.json.

Action	Default binding
Toggle Voice — connects/disconnects voice in the browser	`toggle_voice` (default ⌃V)
Drop Context — sends selected text, clipboard image, or Finder file to Sutando	`drop_context` (default ⌃⇧C)
Toggle Mute — mutes/unmutes microphone during voice	`toggle_mute` (default ⌃M)
Drop Video Clip — sends a screen recording of the active window/screen to Sutando	`drop_video_clip` (default ⌃⇧R)
Drop Screenshot — sends a screenshot of the active window/screen to Sutando	`drop_screenshot` (default ⌃S)

The action names above are the stable keys in state/hotkeys.json; the ⌃-combos are only the current defaults and may be remapped, so treat the action — not the keystroke — as the contract.

The menu bar also has Open Core (brings up the selected core CLI terminal) and Open Dashboard (opens the status dashboard at localhost:7844).

On first run:

Grant Accessibility permission to the Sutando app in System Settings → Privacy & Security
Enable Allow JavaScript from Apple Events in Chrome: View → Developer → Allow JavaScript from Apple Events (required for the Toggle Voice hotkey — default ⌃V, see Keyboard shortcuts)

To opt in, compile and launch it separately: cd src/Sutando && swiftc -O -o Sutando main.swift SutandoConfig.swift -framework Cocoa -framework Carbon -framework ApplicationServices -framework AVFoundation, then run ./Sutando. The app and its accessibility helper are not core boot dependencies.

Proactive mode

startup.sh automatically enables proactive mode. Sutando runs an autonomous loop that:

Processes voice tasks and context drops immediately
Runs health checks and auto-fixes failed services
Picks the highest-value improvement work when idle
Learns from your corrections and adapts over time
Notifies you on Discord and voice when it completes autonomous work

It consumes API quota proportional to how much work it finds to do.

Autonomous self-development is enabled by default. To run Sutando in a stable product context without idle-time code evolution, set SUTANDO_SELF_DEVELOPMENT_ENABLED=0 in .env and restart the core. Sutando continues to process owner requests, monitor health, and deliver tasks; it only stops choosing and executing autonomous improvement work. An explicit owner-requested code change is still allowed.

Security

🚨 Sutando has deep access to your computer — file system, screen, keyboard, browser, email, and phone. Understand the risks before deploying.

Built-in protections:

STIR/SHAKEN verification — inbound calls are checked for carrier-level caller ID attestation. Spoofed numbers are automatically downgraded and denied owner access.
3-tier access control — owner, verified, and unverified callers get different levels of access on phone, Discord, and Telegram.

Recommended setup:

Keep your Twilio phone number private
Set VERIFIED_CALLERS explicitly in .env (don't leave it empty)

macOS permissions Sutando needs (System Settings → Privacy & Security):

Screen Recording → add your selected core CLI (claude or codex) and node. Required for describe_screen, capture_screen, and the screen-capture server (port 7845) — lets Sutando see what you're looking at when you ask "what's on my screen?". Also used by the screen-record skill for subtitled recordings.
Accessibility → add the Sutando menu-bar app. Required for the global hotkeys (see Keyboard shortcuts) and for the macos-use skill to click/type into native apps on your behalf.
Microphone → Chrome (and Terminal, for the screen-record skill). Chrome asks on first voice connect — click Allow.
Contacts / Calendar / Reminders → asked on demand by the features that use them (contact lookup before a call, gws calendar +agenda, reminders.py add/list/complete). You can grant these when first prompted rather than up front.

See SECURITY.md for full details, best practices, and how to test your setup.

For setup guides, operator runbooks, architecture, protocols, and release policy, start at the documentation hub.

Contributing

This is alpha software. The biggest need is testing — try a capability, report what breaks.

Join the Discord for help, discussion, and updates
Open an issue for bugs

How it was built

Sutando was largely built by its own autonomous build loop -- a Claude Code session that reads a build log, picks the highest-value missing piece, builds it, and loops. The human provides direction and testing; the agent does the rest.

Acknowledgments

Voice agent built on bodhi-realtime-agent, a Gemini Live voice session library.

License

MIT

File tree

1257

sonichi/sutandoView on GitHub

.coveragerc
.env.example
.githooks/
  pre-commit
.github/
  dependabot.yml
  ISSUE_TEMPLATE/
    bug_report.md
    feature_request.md
    use-case-submission.md
  PULL_REQUEST_TEMPLATE.md
  workflows/
    bundle-smoke.yml
    ci.yml
    cla-recheck-on-push.yml
    codeql.yml
    coverage-comment.yml
    coverage-gate.yml
    eslint.yml
    lint-agents-md-sync.yml
    lint-claude-home-path.yml
    lint-hermetic-bridge-tests.yml
    lint-host-label.yml
    lint-hotkey-assertions.yml
    lint-src-map.yml
    lint-sutando-home-path.yml
    lint-workspace-resolution.yml
    publish-sparrow.yml
    python39-compat.yml
    regen-wire-list.yml
    ruff.yml
    shellcheck.yml
    workspace-leak-check.yml
.gitignore
AGENTS.md
CHANGELOG-PENDING.md
CHANGELOG.md
CLA.md
CLAUDE.md
CONTRIBUTING.md
docs/
  adr/
    0001-pointer-teacher-brain.md
  architecture-boundaries.md
  avatar-default.html
  built-in-tools.md
  catalog.json
  codex-core.md
  community-use-cases/
    README.md
    self-healing-install.md
    thumbnails/
      .gitkeep
    videos/
      .gitkeep
  design-core-health-verdict.md
  design-credential-capability-resolver.md
  design-mediated-capability-layer.md
  host-cli-bindings.md
  observability/
    example-events.jsonl
    example-realtime-events.jsonl
    example-realtime-usage.jsonl
    example-usage.jsonl
  pointer-teacher-design.md
  README.md
  regen-wire-list-setup.md
  release-process.md
  remote-gateway-protocol.md
  runtime/
  runtime-dependencies.md
    claude-hook-contract-v1.md
  slack-bridge.md
  src-map.md
  state-sync-allowlist.md
  sutando-config.schema.json
  testing-coverage.md
  voice-agent-test-framework.md
  workspace-config.md
  workspace-contract.md
  workspace-design.md
  workspace-hosts-convention.md
  workspace-per-host-paths.md
  workspace-sync.md
eslint.config.mjs
hooks/
  activity-emitter.py
  context-source-guard.py
  gmail-write-guard.py
  human-action-bridge.py
  README.md
  result-file-marker-guard.py
  skill-usage-telemetry.py
  skip-ask-user-question.py
KNOWN_ISSUES.md
LICENSE
package-lock.json
package.json
packages/
  ag2-sparrow/
    .gitignore
    ag2_sparrow/
      __init__.py
      __main__.py
      _dirs.py
      chat_secret_filter.py
      event_channel.py
      event_consumer.py
      event_inbox.py
      gateway_credentials.py
      human_action.py
      local_task_protocol.py
      remote_gateway_bridge.py
      result_markers.py
      result_ready.py
      send_allowlist.py
      task_archive.py
      workspace_lock.py
    pyproject.toml
    README.md
    tests/
      test_ack_retry.py
      test_dns_timeout.py
      test_event_consumer.py
      test_event_inbox_channel.py
      test_event_wiring.py
      test_gateway_status.py
      test_human_action.py
      test_inflight_recovery.py
      test_launch_diagnostics.py
      test_vault_token_tier.py
      test_write_task_atomic.py
    tools/
      sync_from_src.py
      test_no_drift.py
PERSONAL_CLAUDE.md.example
pnpm-workspace.yaml
README.md
REVIEW.md
ruff.toml
schemas/
  skill-manifest.schema.json
scripts/
  agents-md-sync.sh
  browser-transport-build.mjs
  build-bundle.mjs
  check-python39-compat.py
  check-utc-z-strftime.py
  coverage-gate.sh
  cron-gate.sh
  dedup-conversation-store.py
  gen-src-map.py
  hermetic-workspace-guard.py
  import-conversation-log.py
  import-session-metrics.py
  install-git-hooks.sh
  install-personal-claude-hook.sh
  install-session-start-hook.sh
  install-workspace-sync.sh
  lint-class-rules.py
  lint-claude-home-path.sh
  lint-hermetic-bridge-tests.py
  lint-host-label.sh
  lint-hotkey-assertions.sh
  lint-skill.py
  lint-skill.test.py
  lint-sutando-home-path.sh
  lint-vault-sync-paths.sh
  lint-workspace-resolution.sh
  poc-codeql-33-cmd-injection.sh
  poc-codeql-38-url-substring.sh
  poc-codeql-42-39-string-escaping.sh
  poc-codeql-path-injection.sh
  poc-voice-metrics-reset.js
  posthog-usage-report.py
  pr_flag.py
  presenter-mode.sh
  probe-team-sandbox.sh
  python-binary.sh
  query-conversation.sh
  regen-wire-list.py
  restart-voice-agent.sh
  results-health.sh
  review-checks.py
  review-checks.sh
  review-preflight.py
  smoke-bundle.sh
  sutando-config-hooks.sh
  sutando-config.sh
  sutando-migrate.sh
  sutando-shell-setup.sh
  sutando-whoami.sh
  sweep-stranded-claims.sh
  sync-memory.sh
  sync-workspace.sh
  tail-events.sh
  tailscale-serve-voice.sh
  test-event-log.sh
  test-migrate-bundle.sh
  test-pr354-retention-sweep.sh
  test-results-health.sh
  test-x-search-bearer.sh
  validate-access-tiers.py
  voice-lock.py
SECURITY.md
shared/
  ag2_gateway_credentials.py
skills/
  agent-registry/
    hooks/
      session-start.sh
    manifest.json
    scripts/
      _workspace_resolve.py
      registry-client.py
      registry-service.py
    SKILL.md
  agent-room-ops/
    _gateway.py
    authz.py
    doc.py
    events_acceptance.py
    events.py
    gateway_credentials.py
    grant.py
    join.py
    manifest.json
    media.py
    mention.py
    react.py
    read.py
    resolve.py
    room_ops.py
    room-ops-gate.json.example
    rooms.py
    SKILL.md
    test_room_ops.py
    verify_platform_card.py
  audio-transcribe/
    manifest.json
    scripts/
      transcribe.py
    SKILL.md
  bot2bot-post/
    manifest.json
    post.py
    SKILL.md
  call-diagnostics/
    manifest.json
    scripts/
      analysis.py
      diagnose.py
    SKILL.md
  claude-codex/
    manifest.json
    scripts/
      codex-bounded.sh
      codex-run.sh
      review-pr.sh
    SKILL.md
  claude-gemini/
    manifest.json
    scripts/
      gemini-run.sh
    SKILL.md
  claude-router/
    manifest.json
    scripts/
      route-ai.sh
    SKILL.md
  context-drop/
    .gitignore
    build.sh
    manifest.json
    Package.swift
    SKILL.md
    Sources/
      ax-read/
        main.swift
  context-reconstruct/
    SKILL.md
  db-browser/
    manifest.json
    SKILL.md
  deal-finder/
    .gitignore
    manifest.json
    scripts/
      scan.py
    SKILL.md
    state/
      criteria.json
  doc-ingest/
    scripts/
      ingest.py
    SKILL.md
  electron-overlay-dimming/
    manifest.json
    SKILL.md
  email-find/
    manifest.json
    SKILL.md
  gemini-tts/
    manifest.json
    scripts/
      synthesize.sh
    SKILL.md
  gws-gmail-voice/
    manifest.json
    SCORING_RUNBOOK.md
    tools.ts
  image-generation/
    manifest.json
    README.md
    scripts/
      generate.py
    SKILL.md
  install.sh
  learned-skills/
    scripts/
      detect-learned-skills.py
  macos-tools/
    manifest.json
    README.md
    scripts/
      calendar-reader.py
      contacts.py
      email-sender.py
      reminders.py
      screen-capture.sh
    SKILL.md
  macos-use/
    manifest.json
    scripts/
      build.sh
      install-mcp.sh
    SKILL.md
  make-viral-video/
    manifest.json
    scripts/
      asset_cache.py
      build.sh
      codex-prompt.md
      render.py
      test_source_tag.py
      validate_asset.py
      verify_video.py
    SKILL.md
  MANIFEST.md
  media-transcript/
    manifest.json
    scripts/
      transcript.py
    SKILL.md
  meeting-prep/
    manifest.json
    SKILL.md
  meeting-scheduler/
    manifest.json
    scripts/
      policy.py
      schedule_meeting.py
    SKILL.md
  morning-briefing/
    manifest.json
    SKILL.md
  ms365-connect/
    manifest.json
    requirements.txt
    scripts/
      ms365.py
    SKILL.md
  observe/
    observe_policy.py
    SKILL.md
  obsidian-vault/
    manifest.json
    scripts/
      dream.py
    SKILL.md
    tools.ts
  open-sutando-ref/
    manifest.json
    scripts/
      resolve.py
    SKILL.md
  openai-tts/
    manifest.json
    scripts/
      synthesize.sh
    SKILL.md
  overlay-apps/
    app/
      .gitignore
      control-server.js
      main.js
      package-lock.json
      package.json
      preload.js
      stats-renderer.js
... 1019 more

FAQ

sutando is a Claude Code plugin with 57 hand-picked skills for automation work, indexed on Flowy. Install it with the command on its page. It includes agent-registry, agent-room-ops, audio-transcribe. Its skills do not fire on their own yet. Request auto-invocation to have Flowy route them as you prompt. Free and open source.

What is sutando?: My AI Stand — Realtime by Day, Rewriting Itself by Night. Summon my AI superpower. Voice, vision, screen, meetings, calls when I'm engaged. Learns my patterns, ships its own code when I'm not. Runs across my Macs, interacts with people & their Stands.
How do I install sutando?: Run these in Claude Code: npx -y skills add sonichi/sutando --agent claude-code. Then prompt normally.
Does sutando auto-invoke its skills?: Not yet. It is indexed on Flowy as a plain plugin. Request auto-invocation on its page and Flowy will route its skills for you as you prompt.
Is sutando free?: Yes. Flowy is free and open source, with nothing gated. You can read every skill in full before you install.

Automation

Plugin

sutando

FLOW.md

sonichi@sonichi

GitHub

Plugin Signal89/100Strong

maintained100

capable87

documented100

adopted55

Install

$ npx -y skills add sonichi/sutando --agent claude-code

Run the curl in your terminal, the rest in Claude Code.

Stats

TypeScript JavaScript Node Bash Swift

+7 tags

Git GitHub Anthropic OpenAI Gemini Slack Obsidian

365

Stars

Forks

Active

Maintenance

Python

Language

MIT

License

48m ago

Last commit

4mo ago

Created

Repo: sonichi/sutando

In this plugin

57 skills 7 hooks

See everything inside

What's inside

Skills

Show all 57 skills

Hooks

Show all 7 hooks

skip-ask-user-question.pyHook script.

Badges

README

Read on GitHub

Sutando

My AI Stand — Realtime by Day, Rewriting Itself by Night. Summon my AI superpower.

Voice, vision, screen, meetings, calls when I'm engaged. Learns my patterns, ships its own code when I'm not. Runs across my Macs, interacts with people & their Stands.

It belongs entirely to you.

🛠 Open source: this repo — clone, build, run locally on your own Mac. 🍎 Native app preview: sutando.ai — packaged Mac app, request access.

No pay-per-token core API key required. Sutando runs through the Claude Code or Codex CLI session you select, with optional service credentials only for the capabilities you enable.

Named after Stands from JoJo's Bizarre Adventure — a personal spirit that fights on your behalf. Like a Stand, Sutando starts unnamed. As it learns your style and earns real capabilities, it names itself and generates its own avatar — your Stand, unique to you.

https://github.com/user-attachments/assets/a86ec34e-3b26-4011-824c-d2d124753c25

24 tool calls. 6 tasks. 7 minutes. All by voice from a phone. Demo by @liususan091219. Watch on YouTube (full quality) →

⭐ If Sutando is useful to you, star this repo — it's how other people find it.


🤖 The AI agent that presented live	📺 ep.007 — 50 days. 600+ PRs. #1 trending	📺 Vision talk at UC Berkeley

Sutando in action

@sutando-ai channel · Sutando WIRE playlist

What can you do with it?

Talk while you work. You're looking at a doc. You say "make this paragraph shorter." Sutando sees your screen, rewrites the paragraph, and replaces the original text directly.

Reach you anywhere. Voice, Zoom, Google Meet, Telegram, Discord (text + voice channel), web, phone, or email — same agent, same memory, any channel.

Status: Alpha

This is an early-stage project. Honest status:

	Count	Details
Verified working	30	Voice, screen capture, notes, calendar, reminders, contacts, browser, phone calls, meeting dial-in, task delegation, pattern detection, health check, dashboard, Telegram, Discord, multi-machine migration, onboarding tutorial, and more
Needs external setup	3	Twilio (phone), Telegram bot, Discord bot

We're looking for contributors to help test and harden these capabilities. If you try something and it breaks, open an issue.

How it works

See Sutando architecture boundaries for the normative definitions of core, adapters, apps, skills, tooling, and workspace state.

    You ──voice (browser)──► Voice agent ─────────┐
     │                       (Gemini Live,        │
     │                        WS on :9900)        ├──► inline tools (instant,
     │                                            │    in-process: describe_screen,
     ├──phone (Twilio)─────► Conversation server ─┤    get_current_time, hang_up,
     │                       (Gemini Live,        │    dtmf, ...)
     │                        WS on :3100)        │
     │                                            └──┐
     │                                               │   file bridge       .──────▶────────.
     ├──telegram──────────► Telegram bridge ─────────┼── tasks/ ─────────► |               |
     │                                               │                    |   Core        |
     │                                               │                    |   agent ↻     |
     └──discord───────────► Discord bridge ──────────┘                    |               |
                                                                           `──────◀────────'
                                                                                  │
                                                                                  ▼
                                                                          uses anything:
                                                                          email, calendar,
                                                                          browser, files,
                                                                          phone, reminders...
                                    ◄── results/ ◄────────────────────────────────┘
                                (spoken via voice/phone,
                                 text via Telegram/Discord)

    ↻ = a cron job fires the `/proactive-loop` skill every 15 minutes
        (`*/15 * * * *` in the per-host `crons.json`). The skill
        runs as a 10-minute pass that keeps a persistent watcher on
        `tasks/` via Claude Code's `Monitor` tool — pending tasks are
        processed the moment they arrive, not just on the cron tick.
        Each pass also runs health checks and picks the next build-log
        item autonomously.

Four processes work together:

Voice agent (Gemini Live, WebSocket on :9900) — listens and talks in real time for browser voice.
Web client (com.sutando.web-client.plist, HTTP on :8080) — separate launchd service that serves the browser UI. The browser then connects directly to the voice agent's WebSocket on :9900 — the web client is not in the WebSocket data path.
Conversation server (Gemini Live, Twilio WebSocket on :3100) — same role as the voice agent for inbound and outbound phone calls.
Core agent (Claude Code or Codex CLI) — executes tasks with full system access. The persistent CLI session provides an interactive terminal and runs Sutando's task watcher and scheduled work.

Quick start

Prerequisites — bash src/startup.sh checks that these are installed and refuses to boot otherwise:

Claude Code or Codex CLI — whichever you select
Node.js (brew install node)
Python 3 (brew install python3)
fswatch (brew install fswatch) — auto-installs via Homebrew on first start

Recommended, but not checked at boot — macOS 15+ and Node.js 22+.

bash src/verify-setup.sh covers this second list — it checks the Node version and whether your CLI is actually authenticated. Run it if startup succeeds but the core doesn't.

Optional — each unlocks one feature and degrades alone:

Gemini API key — voice (text/core paths work without it)
pip3 install discord.py / slack_bolt — Discord / Slack bridges (Telegram needs no package)
ffmpeg (brew install ffmpeg) — subtitle-burn, video-concat, recording handoff
tmux (brew install tmux) — Sutando.app watcher auto-restart; the core starts without it
git — vault sync, self-upgrade, commit provenance
Twilio account + ngrok — phone calls and SMS

Full list with the line enforcing each, plus what to vendor when embedding Sutando in another application: External runtime dependencies.

# Clone
git clone https://github.com/sonichi/sutando.git
cd sutando

# Configure optional integrations (skip for text/core-only use)
cp .env.example .env
# Add GEMINI_API_KEY only if you want voice

# Start everything
bash src/startup.sh

Why Sutando runs with elevated permissions. Autonomous voice-driven work means startup.sh launches the selected core CLI with unattended approvals and full local access — permission prompts would otherwise break the voice-in / answer-out flow. In exchange:

It's local. Sutando runs entirely on your Mac. No remote control plane, no third party with write access.

You control the audience. 3-tier access gating means owner / verified / unverified callers get different capability bands on phone, Discord, and Telegram. Set VERIFIED_CALLERS in .env before going live.

Actions are auditable. Every task lands in tasks/ + results/, and service activity is written to logs/*.log. Use the Core CLI terminal while it works to watch in real time.

Hooks are your brake pedal. git-rules-guard.sh (see $CLAUDE_CONFIG_DIR/hooks) pops a Discord approval DM for any public write (push / PR / issue comment) regardless of transport. Reject with 👎 to block.

Keep the Core CLI terminal reachable — quota exhaustion or an unrecognized CLI prompt can leave the core agent waiting for you to respond. See Codex core setup to select or roll back the Codex runtime.

macOS permissions — on first run, macOS will ask you to grant Screen Recording, Accessibility, and Microphone access. See Security for what each permission is used for.

Try saying:

"What's on my screen?" — takes a screenshot and describes it
"Summon my computer to zoom" — opens Zoom with screen sharing, join from your phone
"Join my next meeting" — checks your calendar and joins
"Take a note: my first idea" — saves a searchable note
"Tutorial" — walks you through all capabilities step by step

Verify your setup (optional):

bash src/verify-setup.sh

Troubleshooting:

Browser shows blank page? Services may still be starting — wait 5 seconds and refresh
Microphone not working? Chrome will ask for permission on first connect — click Allow
Voice agent not responding? Check logs/voice-agent.log for errors. Common causes:
- GEMINI_API_KEY not set or invalid in .env — get one at ai.google.dev
- Port 9900 already in use — run lsof -i :9900 to check
npm install failed? Make sure Node.js 22+ is installed: node --version
Gemini 429 errors? Your shell may have a stale GEMINI_API_KEY overriding .env — run unset GEMINI_API_KEY then restart
Screen recording produces 0-second files? screencapture -v needs a TTY. Sutando uses ffmpeg instead — make sure it's installed: brew install ffmpeg
Something broke? Run bash src/restart.sh — this kills all services and restarts fresh
Sutando acting confused, contradicting itself, or giving stale answers after a long session? Restart the selected core CLI session to reset its context.
Still stuck? Join the official Discord — real humans and community-run agents answer support questions there.
Phone call answers with "We are sorry, an error has occurred"? The conversation server (skills/phone-conversation/scripts/conversation-server.ts, port 3100) isn't running. Run bash src/startup.sh or bash src/restart.sh to relaunch all services.

Shutting down:

bash src/restart.sh    # stops all services (voice agent, web client, API, bridges, etc.)
pkill -x Sutando # stop the menu bar app

Exiting startup.sh alone does NOT stop background services. Always use restart.sh (or kill-all.sh if available) to cleanly shut everything down.

Uninstalling:

Stop all services: bash src/restart.sh && pkill -x Sutando
Remove the repo: rm -rf ~/Desktop/sutando (or wherever you cloned it)
Remove config: rm -rf $CLAUDE_CONFIG_DIR/projects/*sutando*
Remove npm packages (optional): the repo uses local node_modules/ — deleted with the repo
Remove any tools you installed during setup (e.g. imsg, wacli) via the package manager you used to install them.
If you installed the OS-supervised health checks: bash src/install-health-check-launchd.sh --uninstall (idempotent — no-op if not installed).

Optional integrations

These unlock more capabilities. Add to .env when ready:

Integration	What it unlocks	Setup
Gmail	Read/send/search email from voice	`gws auth setup --login` (OAuth, no app password)
Twilio + ngrok	Phone calls, SMS, meeting dial-in, task delegation via phone	twilio.com + `brew install ngrok` (see Running costs)
Telegram	Message Sutando from your phone. First DM auto-enrolls you as owner (trust-on-first-use). Subsequent senders need to be added: edit `$CLAUDE_CONFIG_DIR/channels/telegram/access.json` → `allowFrom` list.	Create bot via @BotFather, then `/telegram:configure <token>`
Discord	Message Sutando from Discord (DM + channel @mentions)	Developer portal, then `/discord:configure <token>`
Claude for Chrome	Browser automation — navigate, read pages, fill forms, interact with web apps	Install extension, log in with the same account as Claude Code
Sutando app (menu bar)	Optional global hotkeys (see Keyboard shortcuts)	Build and launch separately; core startup stays headless
OS-supervised health checks	Detect stuck loops, dead watchers, and queue pileups even when core is unresponsive — macOS notifies you when Sutando is broken	`bash src/install-health-check-launchd.sh` (idempotent; uninstall with `--uninstall`)
Multi-machine workspace sync	Run the same agent identity across Mac mini + MacBook + Mac Studio etc.; memory + notes + state stay consistent via a private git repo you own	Create a private vault repo, set `vault.remote_url` in `sutando.config.local.json`, run `bash scripts/sync-workspace.sh --init` once + cron it. See docs/workspace-sync.md. The legacy `sync-memory.sh` flow is deprecated in v0.3.0 and removed in v0.4.0.

Running costs

One table, organized by capability. Core access comes from the Claude Code or Codex CLI account you select; the remaining services are optional and mostly free-tier-sufficient.

Capability	When you need it	Service required	Cost
Basic (core agent + screen / notes / calendar / reminders / contacts / browser / iMessage)	Always — this is Sutando's baseline	Claude Code or Codex CLI access + macOS	Depends on the selected CLI account. No Gemini key or Google OAuth is required for core operation.
Voice agent (real-time conversation in browser or on phone)	If you want to talk to Sutando	Gemini voice API	Free tier covers normal use (~15 req/min). Heavy use: Gemini paid ~$0.30–$1.30/hr.
Telegram / Discord / WhatsApp (message Sutando from any of these)	If you want non-voice chat from your phone or desktop	Telegram BotFather, Discord developer portal, `wacli` (bundled)	All free for personal use.
Phone calls / summon (remote control)	If you want Sutando to make inbound/outbound calls, or to share its computer screen via Zoom/Google Meet and be controlled by voice from your phone	Twilio phone number + ngrok webhook	Twilio ~$1/mon number + ~$0.0085/min inbound + ~$0.015/min outbound + Media Streams ~$0.004/min. ngrok and Zoom free tiers both work for the summon flow.
Agent joining meetings via dial-in (PSTN join into Zoom / Google Meet)	If you want the phone agent to dial into a meeting as a participant	Zoom Pro OR Google Workspace Business on the host side (the meeting organizer's account needs toll dial-in enabled)	Zoom Pro ~$15/mon, Google Workspace Business Starter ~$7/mon. Sutando's side is already covered by the Phone row above.

What's inside

Capability	Script	Status
Voice conversation	`voice-agent.ts`	Verified
Task delegation (voice → core)	`task-bridge.ts` + `watch-tasks-stream.sh` + `tasks/` dir	Verified
Screen capture + analysis	`macos-tools` skill	Verified
Notes / second brain	`notes/` directory (YAML-frontmatter markdown)	Verified
Context drop + shortcuts	`src/Sutando/` menu bar app	Verified
Gmail read/send/search	`gws-gmail` skill	Verified
Calendar reading	`google-calendar` skill	Verified
Reminders management	`macos-tools` skill	Verified
Contacts lookup	`macos-tools` skill	Verified
Browser automation	`browser.mjs` + MCP tools	Verified
Conversational phone calls	`phone-conversation/`	Verified (needs Twilio + ngrok)
Phone → task delegation	`phone-conversation/`	Verified (needs Twilio + VERIFIED_CALLERS)
Join Zoom (computer audio)	`inline-tools.ts`	Verified
Join Google Meet (browser audio)	`inline-tools.ts`	Verified
Meeting dial-in (Meet + Zoom)	`phone-conversation/`	Verified (needs Twilio + ngrok)
Meeting approval via Telegram	`phone-conversation/`	Verified (needs Twilio + Telegram)
Inbound call handling	`phone-conversation/`	Verified (needs Twilio)
Telegram messaging	`telegram-bridge.py`	Verified (text + photos + files + voice)
Discord messaging	`discord-bridge.py`	Verified (DMs + channel @mentions + files)
Cross-device task submission	`agent-api.py`	Verified
Health monitoring	`health-check.py`	Verified
Pattern detection + user modeling	Core memory files + selected CLI	Verified
System dashboard	`dashboard.py`	Verified
Info-radar (arXiv / GitHub / HN / news monitoring)	`info-radar` skill + daily digest	Verified
Menu-bar avatar states (idle/listening/speaking/working)	`src/Sutando/main.swift` + `/sse-status`	Verified

Services

When running, Sutando exposes these local ports:

Port	What
8080	Voice web client — talk to Sutando here
7844	Dashboard — status, activity, and capability matrix
7843	Agent API — submit tasks from any device
9900	Voice agent WebSocket
7845	Screen capture server
3100	Phone conversation server (Twilio webhook target)
4040	ngrok admin UI (when ngrok is running)

Keyboard shortcuts

Action	Default binding
Toggle Voice — connects/disconnects voice in the browser	`toggle_voice` (default ⌃V)
Drop Context — sends selected text, clipboard image, or Finder file to Sutando	`drop_context` (default ⌃⇧C)
Toggle Mute — mutes/unmutes microphone during voice	`toggle_mute` (default ⌃M)
Drop Video Clip — sends a screen recording of the active window/screen to Sutando	`drop_video_clip` (default ⌃⇧R)
Drop Screenshot — sends a screenshot of the active window/screen to Sutando	`drop_screenshot` (default ⌃S)

The menu bar also has Open Core (brings up the selected core CLI terminal) and Open Dashboard (opens the status dashboard at localhost:7844).

On first run:

Grant Accessibility permission to the Sutando app in System Settings → Privacy & Security
Enable Allow JavaScript from Apple Events in Chrome: View → Developer → Allow JavaScript from Apple Events (required for the Toggle Voice hotkey — default ⌃V, see Keyboard shortcuts)

Proactive mode

startup.sh automatically enables proactive mode. Sutando runs an autonomous loop that:

Processes voice tasks and context drops immediately
Runs health checks and auto-fixes failed services
Picks the highest-value improvement work when idle
Learns from your corrections and adapts over time
Notifies you on Discord and voice when it completes autonomous work

It consumes API quota proportional to how much work it finds to do.

Security

🚨 Sutando has deep access to your computer — file system, screen, keyboard, browser, email, and phone. Understand the risks before deploying.

Built-in protections:

STIR/SHAKEN verification — inbound calls are checked for carrier-level caller ID attestation. Spoofed numbers are automatically downgraded and denied owner access.
3-tier access control — owner, verified, and unverified callers get different levels of access on phone, Discord, and Telegram.

Recommended setup:

Keep your Twilio phone number private
Set VERIFIED_CALLERS explicitly in .env (don't leave it empty)

macOS permissions Sutando needs (System Settings → Privacy & Security):

Screen Recording → add your selected core CLI (claude or codex) and node. Required for describe_screen, capture_screen, and the screen-capture server (port 7845) — lets Sutando see what you're looking at when you ask "what's on my screen?". Also used by the screen-record skill for subtitled recordings.
Accessibility → add the Sutando menu-bar app. Required for the global hotkeys (see Keyboard shortcuts) and for the macos-use skill to click/type into native apps on your behalf.
Microphone → Chrome (and Terminal, for the screen-record skill). Chrome asks on first voice connect — click Allow.
Contacts / Calendar / Reminders → asked on demand by the features that use them (contact lookup before a call, gws calendar +agenda, reminders.py add/list/complete). You can grant these when first prompted rather than up front.

See SECURITY.md for full details, best practices, and how to test your setup.

For setup guides, operator runbooks, architecture, protocols, and release policy, start at the documentation hub.

Contributing

This is alpha software. The biggest need is testing — try a capability, report what breaks.

Join the Discord for help, discussion, and updates
Open an issue for bugs

How it was built

Acknowledgments

Voice agent built on bodhi-realtime-agent, a Gemini Live voice session library.

License

MIT

File tree

1257

sonichi/sutandoView on GitHub

.coveragerc
.env.example
.githooks/
  pre-commit
.github/
  dependabot.yml
  ISSUE_TEMPLATE/
    bug_report.md
    feature_request.md
    use-case-submission.md
  PULL_REQUEST_TEMPLATE.md
  workflows/
    bundle-smoke.yml
    ci.yml
    cla-recheck-on-push.yml
    codeql.yml
    coverage-comment.yml
    coverage-gate.yml
    eslint.yml
    lint-agents-md-sync.yml
    lint-claude-home-path.yml
    lint-hermetic-bridge-tests.yml
    lint-host-label.yml
    lint-hotkey-assertions.yml
    lint-src-map.yml
    lint-sutando-home-path.yml
    lint-workspace-resolution.yml
    publish-sparrow.yml
    python39-compat.yml
    regen-wire-list.yml
    ruff.yml
    shellcheck.yml
    workspace-leak-check.yml
.gitignore
AGENTS.md
CHANGELOG-PENDING.md
CHANGELOG.md
CLA.md
CLAUDE.md
CONTRIBUTING.md
docs/
  adr/
    0001-pointer-teacher-brain.md
  architecture-boundaries.md
  avatar-default.html
  built-in-tools.md
  catalog.json
  codex-core.md
  community-use-cases/
    README.md
    self-healing-install.md
    thumbnails/
      .gitkeep
    videos/
      .gitkeep
  design-core-health-verdict.md
  design-credential-capability-resolver.md
  design-mediated-capability-layer.md
  host-cli-bindings.md
  observability/
    example-events.jsonl
    example-realtime-events.jsonl
    example-realtime-usage.jsonl
    example-usage.jsonl
  pointer-teacher-design.md
  README.md
  regen-wire-list-setup.md
  release-process.md
  remote-gateway-protocol.md
  runtime/
  runtime-dependencies.md
    claude-hook-contract-v1.md
  slack-bridge.md
  src-map.md
  state-sync-allowlist.md
  sutando-config.schema.json
  testing-coverage.md
  voice-agent-test-framework.md
  workspace-config.md
  workspace-contract.md
  workspace-design.md
  workspace-hosts-convention.md
  workspace-per-host-paths.md
  workspace-sync.md
eslint.config.mjs
hooks/
  activity-emitter.py
  context-source-guard.py
  gmail-write-guard.py
  human-action-bridge.py
  README.md
  result-file-marker-guard.py
  skill-usage-telemetry.py
  skip-ask-user-question.py
KNOWN_ISSUES.md
LICENSE
package-lock.json
package.json
packages/
  ag2-sparrow/
    .gitignore
    ag2_sparrow/
      __init__.py
      __main__.py
      _dirs.py
      chat_secret_filter.py
      event_channel.py
      event_consumer.py
      event_inbox.py
      gateway_credentials.py
      human_action.py
      local_task_protocol.py
      remote_gateway_bridge.py
      result_markers.py
      result_ready.py
      send_allowlist.py
      task_archive.py
      workspace_lock.py
    pyproject.toml
    README.md
    tests/
      test_ack_retry.py
      test_dns_timeout.py
      test_event_consumer.py
      test_event_inbox_channel.py
      test_event_wiring.py
      test_gateway_status.py
      test_human_action.py
      test_inflight_recovery.py
      test_launch_diagnostics.py
      test_vault_token_tier.py
      test_write_task_atomic.py
    tools/
      sync_from_src.py
      test_no_drift.py
PERSONAL_CLAUDE.md.example
pnpm-workspace.yaml
README.md
REVIEW.md
ruff.toml
schemas/
  skill-manifest.schema.json
scripts/
  agents-md-sync.sh
  browser-transport-build.mjs
  build-bundle.mjs
  check-python39-compat.py
  check-utc-z-strftime.py
  coverage-gate.sh
  cron-gate.sh
  dedup-conversation-store.py
  gen-src-map.py
  hermetic-workspace-guard.py
  import-conversation-log.py
  import-session-metrics.py
  install-git-hooks.sh
  install-personal-claude-hook.sh
  install-session-start-hook.sh
  install-workspace-sync.sh
  lint-class-rules.py
  lint-claude-home-path.sh
  lint-hermetic-bridge-tests.py
  lint-host-label.sh
  lint-hotkey-assertions.sh
  lint-skill.py
  lint-skill.test.py
  lint-sutando-home-path.sh
  lint-vault-sync-paths.sh
  lint-workspace-resolution.sh
  poc-codeql-33-cmd-injection.sh
  poc-codeql-38-url-substring.sh
  poc-codeql-42-39-string-escaping.sh
  poc-codeql-path-injection.sh
  poc-voice-metrics-reset.js
  posthog-usage-report.py
  pr_flag.py
  presenter-mode.sh
  probe-team-sandbox.sh
  python-binary.sh
  query-conversation.sh
  regen-wire-list.py
  restart-voice-agent.sh
  results-health.sh
  review-checks.py
  review-checks.sh
  review-preflight.py
  smoke-bundle.sh
  sutando-config-hooks.sh
  sutando-config.sh
  sutando-migrate.sh
  sutando-shell-setup.sh
  sutando-whoami.sh
  sweep-stranded-claims.sh
  sync-memory.sh
  sync-workspace.sh
  tail-events.sh
  tailscale-serve-voice.sh
  test-event-log.sh
  test-migrate-bundle.sh
  test-pr354-retention-sweep.sh
  test-results-health.sh
  test-x-search-bearer.sh
  validate-access-tiers.py
  voice-lock.py
SECURITY.md
shared/
  ag2_gateway_credentials.py
skills/
  agent-registry/
    hooks/
      session-start.sh
    manifest.json
    scripts/
      _workspace_resolve.py
      registry-client.py
      registry-service.py
    SKILL.md
  agent-room-ops/
    _gateway.py
    authz.py
    doc.py
    events_acceptance.py
    events.py
    gateway_credentials.py
    grant.py
    join.py
    manifest.json
    media.py
    mention.py
    react.py
    read.py
    resolve.py
    room_ops.py
    room-ops-gate.json.example
    rooms.py
    SKILL.md
    test_room_ops.py
    verify_platform_card.py
  audio-transcribe/
    manifest.json
    scripts/
      transcribe.py
    SKILL.md
  bot2bot-post/
    manifest.json
    post.py
    SKILL.md
  call-diagnostics/
    manifest.json
    scripts/
      analysis.py
      diagnose.py
    SKILL.md
  claude-codex/
    manifest.json
    scripts/
      codex-bounded.sh
      codex-run.sh
      review-pr.sh
    SKILL.md
  claude-gemini/
    manifest.json
    scripts/
      gemini-run.sh
    SKILL.md
  claude-router/
    manifest.json
    scripts/
      route-ai.sh
    SKILL.md
  context-drop/
    .gitignore
    build.sh
    manifest.json
    Package.swift
    SKILL.md
    Sources/
      ax-read/
        main.swift
  context-reconstruct/
    SKILL.md
  db-browser/
    manifest.json
    SKILL.md
  deal-finder/
    .gitignore
    manifest.json
    scripts/
      scan.py
    SKILL.md
    state/
      criteria.json
  doc-ingest/
    scripts/
      ingest.py
    SKILL.md
  electron-overlay-dimming/
    manifest.json
    SKILL.md
  email-find/
    manifest.json
    SKILL.md
  gemini-tts/
    manifest.json
    scripts/
      synthesize.sh
    SKILL.md
  gws-gmail-voice/
    manifest.json
    SCORING_RUNBOOK.md
    tools.ts
  image-generation/
    manifest.json
    README.md
    scripts/
      generate.py
    SKILL.md
  install.sh
  learned-skills/
    scripts/
      detect-learned-skills.py
  macos-tools/
    manifest.json
    README.md
    scripts/
      calendar-reader.py
      contacts.py
      email-sender.py
      reminders.py
      screen-capture.sh
    SKILL.md
  macos-use/
    manifest.json
    scripts/
      build.sh
      install-mcp.sh
    SKILL.md
  make-viral-video/
    manifest.json
    scripts/
      asset_cache.py
      build.sh
      codex-prompt.md
      render.py
      test_source_tag.py
      validate_asset.py
      verify_video.py
    SKILL.md
  MANIFEST.md
  media-transcript/
    manifest.json
    scripts/
      transcript.py
    SKILL.md
  meeting-prep/
    manifest.json
    SKILL.md
  meeting-scheduler/
    manifest.json
    scripts/
      policy.py
      schedule_meeting.py
    SKILL.md
  morning-briefing/
    manifest.json
    SKILL.md
  ms365-connect/
    manifest.json
    requirements.txt
    scripts/
      ms365.py
    SKILL.md
  observe/
    observe_policy.py
    SKILL.md
  obsidian-vault/
    manifest.json
    scripts/
      dream.py
    SKILL.md
    tools.ts
  open-sutando-ref/
    manifest.json
    scripts/
      resolve.py
    SKILL.md
  openai-tts/
    manifest.json
    scripts/
      synthesize.sh
    SKILL.md
  overlay-apps/
    app/
      .gitignore
      control-server.js
      main.js
      package-lock.json
      package.json
      preload.js
      stats-renderer.js
... 1019 more

FAQ

What is sutando?: My AI Stand — Realtime by Day, Rewriting Itself by Night. Summon my AI superpower. Voice, vision, screen, meetings, calls when I'm engaged. Learns my patterns, ships its own code when I'm not. Runs across my Macs, interacts with people & their Stands.
How do I install sutando?: Run these in Claude Code: npx -y skills add sonichi/sutando --agent claude-code. Then prompt normally.
Does sutando auto-invoke its skills?: Not yet. It is indexed on Flowy as a plain plugin. Request auto-invocation on its page and Flowy will route its skills for you as you prompt.
Is sutando free?: Yes. Flowy is free and open source, with nothing gated. You can read every skill in full before you install.