embedded-ai-harness

The Harness — AI Closed-Loop Programming for Embedded Systems

Spec to silicon, hands off.

A horse is strong, fast, and willing — and useless for heavy loads until you harness it. The harness is not a part of the horse and not a part of the cart: it is the coupling that turns raw strength into pulled weight.

An AI is the same. It can write firmware all day — but it can't flash a board, can't see it boot, can't know whether its fix actually worked on real hardware. Unharnessed, it generates code and hopes. This repository is the harness: strap the AI in, and it pulls — writes the code, compiles it, flashes it onto a real ESP32, tests it against real WiFi, MQTT, BLE and RF, reads the failures, corrects itself, and goes again — until the tests run clean.

🔄 AI Closed-Loop Programming

Today's AI coding is open-loop: prompt → code → hope. No feedback, so errors accumulate uncorrected — which is exactly why people don't trust AI-written firmware. AI Closed-Loop Programming (AICLP) closes the loop with reality:

            FSD  ──────  the setpoint: what "done" means
             │
             ▼
   ┌──── code → build → flash ────┐        forward path
   │                              ▼
   │                        real hardware
   │                              │
   └── correct ◄── tests ◄────────┘        feedback path

        the loop exits when the error signal is zero: tests green

Every embedded engineer knows this diagram — it's a control loop. The spec (FSD) is the setpoint, the firmware on the chip is the plant, the tests are the sensor, failing tests are the error signal, and the AI is the controller that corrects until the error reaches zero.

True TDD, enabled by AI. For twenty-five years, developers drove and tests advised — written after the code, skipped under deadline, tuned until they passed. Here the tests drive for the first time: derived from the spec, run on real silicon, and the only way the AI gets to stop.

🗺️ The Journey — from idea to shipped product

Each phase ends at a gate, derived from project state and never declared — nobody types "phase complete". A gate is not a marker you pass: if anything it requires is unmet, the work loops back to the step that owns it and the whole check runs again. And after shipment the same journey repeats in miniature for every new feature: describe it in a sentence, the loop refuses to code anything no requirement covers, the spec absorbs the delta, and the phases collapse to minutes. No code without a clause is what keeps the spec true for the product's whole life.

🧰 What the Harness consists of

• The method — four Claude Code skills, one per phase: /define (the FSD: atomic, falsifiable requirements, each with its verification contract), /harness (one-time setup), /commission and /build (the loop's driver: test design, the plan, audit, what's next).
• The testbench — a Raspberry Pi test instrument that gives the AI hands and eyes on real hardware. Described below.
• The dev skills — ESP-IDF and PlatformIO lifecycles, logging, WiFi, BLE, MQTT, debugging, RF, CI — the loop's individual muscles.

✅ Prerequisites — what the loop needs

Throughout this README, $BENCH is the bench's IP address — never a .local name, which does not resolve from inside a container. Find it once:

BENCH=$(python3 .claude/skills/esp-idf-handling/discover-testbench.py | jq -r .ip)

Nothing else — and in particular no local ESP-IDF or PlatformIO installation. The forward path runs through GitHub Actions:

git push ─► CI builds in a pinned container ─► gh run download ─► flash to a slot ─► observe

The loop flashes the artefact CI produced, never a binary that happens to be sitting in a local build/ directory. That is not a convenience, it is what makes the evidence mean anything: the bytes on the chip are the bytes in the run you can point at, built by a toolchain whose version is pinned in the workflow rather than whatever is installed on somebody's laptop. The same property is what lets a tag ship — the release-verify job flashes the released artefact to the bench and reruns the journey, so a red journey is a release that does not happen.

The toolchain skills (esp-idf-handling, esp-pio-handling) drive that pipeline and know how to build locally too, which is useful when you are iterating by hand. The loop does not depend on it.

The FSD, tests, firmware, CI and documentation are what the loop produces, not what you bring.

🔌 The Testbench — the loop's hands and eyes

Working on an ESP32 normally means being physically attached to it — and an AI can't hold a USB cable. The testbench puts the boards on a Raspberry Pi and turns everything into HTTP:

 LAN (192.168.0.x)
       |
       | eth0 (wired)
       v
  Raspberry Pi ---- wlan0 (WiFi test AP: 192.168.4.x)
  testbench_7e71     hci0  (Bluetooth LE)
       |             UDP :5555 (log receiver)
       | USB hub (internal on Pi 3/4/5, external on Zero)
       |
  +----+----+----+----+
  |    |    |    |
 :4001 :4002 :4003 :4004  <- auto-assigned (4001 + slot index)
 SLOT1 SLOT2 SLOT3 SLOT4  <- one per detected hub port

• Plug in a board → it's ready. Auto-detected in seconds and mapped to a fixed port by which USB connector it's in — same connector, same port, always. That's slot-based identity: a slot is a physical hole in the hub, so scripts and platformio.ini never go stale when boards swap or the kernel renames /dev/ttyACM0.

• Serial over the network at rfc2217://$BENCH:4001 — esptool, PlatformIO, ESP-IDF and anything on pyserial speak it natively.

• Flash three ways — over the network, locally on the Pi, or over the air.

• Debugging out of the box — OpenOCD starts itself for USB-JTAG chips and GDB connects on that slot's own port (3333 + slot index). Slots are independent: two boards debug at once, each selected by its USB port path, because every ESP32 with built-in JTAG enumerates as 303a:1001 and VID:PID alone cannot say which board is which.

• The Pi is the test equipment. Its WiFi becomes the access point your board joins, its Bluetooth scans and connects, optional SDR and Si5351 hardware receive and transmit on 433 MHz, and boards log to it over UDP when USB is busy.

• It presses the buttons. GPIO wired to reset and boot forces download mode and rescues boot-looping boards with nobody in the room.

• Claude drives all of it through 70 MCP tools or the bundled skills. The newest endpoints — serial write, bench reset, the slot access manager — are HTTP and skills only; the MCP surface has not caught up yet.

• Every slot is recorded, always. A reader runs whether or not anyone is watching, so a boot banner is in the buffer before you think to ask for it, and tcp_port + 1000 is a read-only fan-out of the same bytes for as many watchers as you like.

• It has its own test partner. test-firmware/ is an ESP32 image the bench builds, versions and flashes itself. It is not a device under test — it is the counterpart the bench measures itself against: it joins the test AP, hosts one of its own, answers HTTP, and replies on a serial console. The bench has one radio and cannot be the access point its own station tests join, so the partner is how half these requirements are provable at all — and why the suite never depends on a project's firmware.

Honest limits: one writing serial client per board — RFC2217 gives one session, though any number can read the fan-out — the SDR is one dongle, one user, and the API has no authentication: keep the bench on a network you trust.

🚀 Quick Start — building the bench

You need a Raspberry Pi with onboard WiFi and Bluetooth running Raspberry Pi OS Lite (64-bit). A Pi Zero 2 W also needs a USB hub and a USB Ethernet adapter, since wlan0 is reserved for testing; a Pi 3/4/5 has both built in. An RTL-SDR dongle, an Si5351 + PE4302, and jumper wires to the board's EN/BOOT pins are all optional.

git clone https://github.com/SensorsIot/Embedded-AI-Harness.git
cd Embedded-AI-Harness/pi
sudo bash install.sh

That installs every dependency (pyserial, hostapd, dnsmasq, bleak, esptool, OpenOCD, rtl-sdr/rtl_433, mosquitto), sets up the udev hotplug rules, and starts the portal as a systemd service. Plug in a board and check:

curl http://$BENCH:8080/api/devices | jq

Slots are auto-detected — no config file needed. Create /etc/rfc2217/testbench.json only to rename slots, pin ports, declare GPIO pins, or register an ESP-Prog probe; sudo rfc2217-learn-slots prints one for you.

On a Pi Zero 2 W, do the memory hardening first. With 512 MB the board OOM-crashes under load, and hard crashes corrupt the SD card. See User Manual §2.2.

🔧 Usage

Watch a board boot — no client library, just HTTP:

curl -X POST http://$BENCH:8080/api/serial/reset \
  -H 'Content-Type: application/json' -d '{"slot":"SLOT1"}'

Point your existing tools at it. PlatformIO needs one line (upload_port = rfc2217://$BENCH:4001); esptool takes the same URL, and the binaries stay on your machine:

esptool --port rfc2217://$BENCH:4001 --chip esp32c3 \
  write-flash 0x10000 firmware.bin

Write a test that uses the whole bench — reset the board, give it a network to join, wait for it to appear, then talk to it: