/methodology

name: methodology
version: 0.3.0
description: >
  Analyzes captured HTTP traffic, designs the CLI architecture, and implements the
  Python CLI package (Phase 2): parse raw-traffic.json, identify the protocol, write
  api-spec.json, scaffold from templates, and implement endpoint methods and Click
  command groups. Use after a capture completes and raw-traffic.json exists.
when_to_use: >
  Trigger phrases: "analyze traffic", "design CLI", "implement CLI", "build CLI from
  network traffic", "generate API wrapper", "reverse engineer web API", "start
  Phase 2", or after the capture skill finishes. Not for traffic recording (capture),
  test writing (testing), or quality checks (standards).

CLI-Anything-Web Methodology (Phase 2)

Analyze captured traffic, design the CLI command structure, and implement the complete Python CLI package. This skill owns the core transformation from raw HTTP traffic to a production-ready CLI.

Copy this checklist and check off items as you complete them:

Phase 2 Progress:
- [ ] Prerequisites: raw-traffic.json exists (+ auth state if the site needs auth)
- [ ] Step A: traffic analyzed, protocol identified, <APP>.md written
- [ ] Step A: api-spec.json written (every endpoint cites raw-traffic.json evidence)
       and passes `cli-web-devkit spec validate`
- [ ] Step B.0: scaffolded via scaffold-cli.py (.manifest.json present)
- [ ] Step B: client endpoint methods implemented from the spec
- [ ] Step B: command modules implemented + registered, REPL help in sync
- [ ] Smoke check passed (no protocol leaks), phase-state marked complete

---

Prerequisites (Hard Gate)

Do NOT start unless:

• [ ] `raw-traffic.json` exists (with WRITE operations, or read-only GET-only traffic)
• [ ] Auth state was captured during Phase 1 (if the site requires auth)

If raw-traffic.json is missing or has no WRITE operations, invoke the `capture` skill first. If Phase 1 state shows `failed`, follow `skills/shared/RECOVERY.md` §phase-state Check Failures before re-running.

**Exception for read-only sites:** If the site is genuinely read-only (search engine, dashboard, analytics viewer with no create/update/delete), the trace may contain only GET requests. In this case, note "read-only site — no write operations" in `<APP>.md` and proceed. The generated CLI will have read-only commands (list, get, search) but no create/update/delete commands. This is valid.

**No-auth sites:** If the target site requires no authentication (public API, no login needed), the "Auth state captured" prerequisite does not apply. Note "no-auth site" in `<APP>.md` and proceed.

---

Step A: Analyze (API Discovery)

**Goal:** Map raw traffic to a structured API model.

**Process:**

0. **Read `traffic-analysis.json` first** (if it exists alongside `raw-traffic.json`). This file is auto-generated by `parse-trace.py` or `mitmproxy-capture.py` → `analyze-traffic.py` and contains pre-detected protocol type, auth pattern, endpoint grouping, GraphQL operations, batchexecute RPC IDs, and suggested CLI commands. Use it as a starting point — verify its findings and fill in anything marked "unknown" by reading `raw-traffic.json` manually.

**Enhanced analysis (present only when captured via mitmproxy):** `request_sequence` (timeline-ordered requests with auth-flow detection), `session_lifecycle` (cookie inventory, auth-cookie identification, session pattern), and `endpoint_sizes` (response-size classification). If these are missing (`has_timestamps: false`), the capture came from the default trace path — rely on manual analysis for sequence/session detail.

If `traffic-analysis.json` doesn't exist, run the analyzer:

   python ${CLAUDE_PLUGIN_ROOT}/scripts/analyze-traffic.py \
     <app>/traffic-capture/raw-traffic.json --summary

1. Parse `raw-traffic.json` (for details the analyzer couldn't extract) 2. Group requests by base path (e.g., `/api/v1/boards/`, `/api/v1/items/`) 3. For each endpoint group, identify:

• HTTP method (GET/POST/PUT/DELETE/PATCH)
• URL pattern (extract path parameters like `:id`)
• Query parameters and their types
• Request body schema (JSON fields, types, required/optional)
• Response body schema
• Authentication method (Bearer token, cookie, API key)
• Rate limiting signals (429 responses, retry-after headers)

4. **Identify RPC protocol type** -- classify the API transport:

| Protocol | Detection Signal | Client Pattern | |----------|-----------------|----------------| | REST | Resource URLs (`/api/v1/boards/:id`), standard HTTP methods | `client.py` with method-per-endpoint | | GraphQL | Single `/graphql` endpoint, `query`/`mutation` in body | `client.py` with query templates | | gRPC-Web | `application/grpc-web` content type, binary payloads | Proto-based client | | Google batchexecute | `batchexecute` in URL, `f.req=` body, `)]}'\n` prefix | `rpc/` subpackage (see `references/google-batchexecute.md`) | | Custom RPC | Single endpoint, method name in body, proprietary encoding | Custom codec module | | Public REST API | Documented `/api/` endpoints, OpenAPI spec, JSON responses | Standard `client.py` with httpx | | Plain HTML (no framework) | No SPA root, no framework globals, data in `<table>`/`<div>` | `client.py` with httpx + BeautifulSoup4 |

This determines client architecture in Step B -- REST uses simple `client.py`, non-REST protocols need a dedicated `rpc/` subpackage with encoder/decoder/types.

5. Detect data model:

• Entity types (boards, items, users, projects...)
• Relationships (board has many items, item belongs to board)
• ID formats (UUID, numeric, slug)

6. Detect auth pattern:

• Cookie-based sessions
• Bearer/JWT tokens
• OAuth refresh flow
• API key headers
• Browser-delegated auth: tokens embedded in page JavaScript (e.g., `WIZ_global_data`),

not in HTTP headers. Requires CDP for initial cookies, HTTP for token e