/copilot-sdk-e2e-dev

name: copilot-sdk-e2e-dev
description: Spin up a live local Omnigent server and exercise the GitHub Copilot SDK harness end-to-end — build copilot agents, run real turns, smoke-test, and bug-bash. Load when developing, testing, or debugging the copilot harness (omnigent/inner/copilot_executor.py, copilot_harness.py, omnigent/onboarding/copilot_auth.py) or its auth / model / tool-bridge behavior.

Copilot SDK harness: end-to-end dev & testing

The `copilot` harness drives the **GitHub Copilot SDK** (`github-copilot-sdk`, imported as `copilot`) — a persistent `CopilotClient` + `CopilotSession` per Omnigent conversation — and bridges Omnigent's `sys_*` tools into Copilot as SDK `Tool`s. The Python SDK **bundles the Copilot CLI binary it drives** as a backing server, so there is no separate `@github/copilot` install. This skill is the proven recipe for running it **for real** against a live local server — not just the unit tests.

> The harness runs as a **local runner** from your current checkout, so > `omni run <bundle> --server <url>` exercises exactly the code you're on.

Prerequisites (check these first)

1. **You're on the branch you want to test.** The copilot harness is an optional extra — install it (without disturbing other extras) with `uv sync --frozen --extra dev --extra copilot`. NB: a bare `uv run --frozen --extra dev` re-syncs the venv and **prunes** the copilot SDK; for live testing call `.venv/bin/omni` / `.venv/bin/python` directly and avoid `uv run` mid-session. 2. **The SDK is installed:** `.venv/bin/python -c "import copilot; print(copilot.__file__)"`. 3. **A GitHub token with Copilot access is configured.** Copilot needs a fine-grained PAT with the "Copilot Requests" permission, or an OAuth token from the GitHub CLI / Copilot CLI app (classic `ghp_` PATs are rejected). Verify (booleans only — never print the token):

   .venv/bin/python -c "from omnigent.onboarding.copilot_auth import copilot_github_token_configured; import os; print('config:', copilot_github_token_configured(), 'env:', bool(os.environ.get('GH_TOKEN') or os.environ.get('COPILOT_GITHUB_TOKEN')))"

If both are `False`, run `omni setup` and register a Copilot token, or `export GH_TOKEN=$(gh auth token)` (when `gh` is logged into an account with Copilot). Check the account's entitlement with `gh api /copilot_internal/user` (look for `chat_enabled`/`cli_enabled`). 4. **Network egress to GitHub's Copilot backend.** A turn that hangs or fails to connect on a locked-down host is usually egress, not a harness bug.

Step 1 — start a local server

cd /path/to/omnigent
.venv/bin/omni server --port 7788 --no-open    # foreground; or `omni server --background` for detached
curl -s http://127.0.0.1:7788/health           # {"status":"ok"}

Use the URL below as `$SERVER`.

Step 2 — build a copilot agent bundle

A spec with `spec_version` **must be a directory containing `config.yaml`** — not a single `.yaml` file. Minimal copilot agent:

mkdir -p /tmp/copilot-dev
cat > /tmp/copilot-dev/config.yaml <<'YAML'
spec_version: 1
name: copilot-dev
description: Copilot SDK dev/test agent.
executor:
  type: omnigent
  config:
    harness: copilot
    # model: gpt-5-mini      # optional; omit for Copilot auto-select
prompt: |
  You are a terse test agent. Answer in as few words as possible.
YAML

For sub-agents, tools, guardrails/policies, copy the field shapes from `examples/polly/config.yaml` and `examples/debby/config.yaml`. (Declare policies under `guardrails.policies:` — a top-level `policies:` key is silently dropped on the `spec_version` + `config.yaml` path.)

Step 3 — run a turn (and smoke-test)

SERVER=http://127.0.0.1:7788
timeout 280 .venv/bin/omni run /tmp/copilot-dev \
  -p "Reply with exactly the single word: PONG" \
  --server "$SERVER" 2>&1

A healthy run prints connection lines then the reply (`PONG`). If that works, the full stack is good: token, egress, bundled CLI, harness.

• **Shell / file tools:** add `--tools coding`.
• **Specific model:** add `--model gpt-5-mini` (or `claude-haiku-4.5`, `auto`).

Targeted scenarios

| Goal | How | |------|-----| | Native tools (shell/edit/read) | `--tools coding`, prompt to create→read→edit a file; confirm it actually touches disk | | Bridged `sys_*` / sub-agent dispatch | declare a sub-agent (harness `copilot` so auth is satisfied), prompt the parent to delegate — exercises the SDK `Tool` async-handler bridge into `_tool_executor` | | Model routing | run the same bundle with several `--model` values; an unknown id fails **loud**, a `databricks-*` id is dropped to auto with a warning | | LLM-phase policy | add a guardrail that denies a keyword; confirm `PHASE_LLM_REQUEST`/`PHASE_LLM_RESPONSE` blocks it | | Concurrency / leaks | fire several `omni run … &` at once; then `pgrep -af "copilot/bin/copilot"` to check for orphaned bundled-CLI subprocesses |

Running polly (or any orchestrator) on a copilot brain

The copilot harness can serve as an **async orchestrator** brain (polly / debby), not just a standalone agent — it dispatches to sub-agents via the bridged `sys_*` tools and synthesizes their results. Two ways to exercise it:

**1. Committed regression guard (brain smoke).** `tests/e2e/test_polly_copilot_e2e.py` boots a local server from your checkout and runs `examples/polly` with `--harness copilot --model auto`, asserting the brain boots and replies. It is **skipped** unless a Copilot token is configured (so CI without one skips it). Run it with:

.venv/bin/python -m pytest -o addopts="" tests/e2e/test_polly_copilot_e2e.py -v

**2. Full orchestration (dispatch → collect → synthesize).** Use the `polly-e2e-dev` driver (in the internal `agent-framework` clone) — it boots a local server, polls the AP API, auto-answers elicitations, and asserts the fan-out. Drive the brain on copilot with `--brain-harness copilot`, and **always pass a Cop