/esp-idf-handling

name: esp-idf-handling
description: >
  Complete ESP-IDF lifecycle: project setup, build, flash, monitor, and OTA.
  Automatically detects whether a testbench is available or the device is
  connected locally via USB. Covers sdkconfig, partition tables, esptool,
  RFC2217 remote flashing, GPIO download mode, OTA updates, crash recovery,
  and flapping. Triggers on "flash", "build", "upload", "idf.py", "monitor",
  "serial console", "slot", "testbench", "esptool", "OTA", "erase",
  "download mode", "crash loop", "flapping", "bricked", "menuconfig",
  "set-target", "sdkconfig", "partition".

ESP-IDF Handling

Complete lifecycle for ESP-IDF projects — from project creation to flashing and monitoring. Automatically adapts to local USB or remote testbench.

Step 1: Detect Environment

Determine whether a testbench is available or the device is local.

curl -s $TESTBENCH_URL/api/info

• **Response received** → testbench is available, use remote flashing (RFC2217/OTA)
• **Connection refused / timeout** → try the discovery script:

  sudo python3 .claude/skills/esp-idf-handling/discover-testbench.py --hosts

• **Still no response** → no testbench, use local USB flashing

Step 2: Project Setup

source /opt/esp-idf/export.sh
idf.py create-project <name>           # Create new project
idf.py set-target esp32s3              # Set target chip (esp32, esp32s3, esp32c3, etc.)
idf.py menuconfig                      # Interactive configuration (writes sdkconfig)

sdkconfig.defaults

Put persistent config in `sdkconfig.defaults` (not `sdkconfig` which is generated):

CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y
CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions-4mb.csv"

Step 3: Build

source /opt/esp-idf/export.sh
idf.py build                           # Build
idf.py fullclean                       # Clean build directory

Step 3b: When there is no local toolchain

`export.sh` missing is not a broken setup — a project set up by the [`setup-action`](../setup-action/SKILL.md) skill builds on GitHub deliberately, and some machines never install ESP-IDF at all. Check before assuming:

ls /opt/esp-idf/export.sh ~/esp/esp-idf/export.sh 2>/dev/null || echo "build in CI"

Then the binaries have to come back before Step 4 can run:

# `gh` needs a token, and a tool call is a NON-interactive shell, where
# ~/.bashrc returns before it sources the secrets. Without this line gh says
# "not logged into any GitHub hosts" even though the terminal works fine.
# Never `gh auth login` — see the `github` skill.
. /run/secrets/env 2>/dev/null

gh run download -R <owner>/<repo> -D /tmp/fw            # latest successful run
gh run download -R <owner>/<repo> <run-id> -D /tmp/fw   # one specific run
gh release download v1.2.0 -R <owner>/<repo> -D /tmp/fw # a published release

Without a token, download the artefact zip from the run page in a browser and unpack it — the flash steps below only care that the files exist.

`setup-action`'s template publishes exactly what the two flash paths need:

| Artefact | Feeds | |---|---| | `coldflash/` — `flash_args` plus every image it names | Step 4a/4b — a first flash over USB | | `firmware_v<version>.bin` | Step 4c — the OTA image | | `sdkconfig.generated` | the effective config, since `sdkconfig` is not committed |

**Use the `flash_args` form in Step 4b when the artefact carries one**, and the explicit-offset form only when it does not. Offsets are not constant: enabling OTA moves the app from `0x10000` to `0x20000` and adds `ota_data_initial.bin` at `0xf000`. A remembered offset writes into the wrong partition and boots the previous firmware, which looks exactly like a flash that worked.

Confirm what you are about to flash rather than trusting the filename — the version is compiled into the image:

strings /tmp/fw/coldflash/firmware.bin | grep -m1 '^[0-9]\+\.[0-9]\+\.[0-9]\+'

Flash Size and Partition Tables

**Measure the flash; never assume it.** There is no universal default — ESP-IDF picks one per target, and it is often smaller than expected (an ESP32-C3 build with nothing set comes out at 2 MB). Whatever it picks is written into the image header, and the bootloader then *prints that value back at you*, so a boot log saying `SPI Flash Size : 2MB` is only repeating the config. It is not evidence about the chip, and reading it as such is an easy way to design a partition layout around a number nobody checked.

Getting it wrong is not loud. Configure more than the part has and the partition table extends past the end of flash: the build succeeds, the flash succeeds, and the failure arrives later as corruption at whatever offset first exceeds the physical device — usually OTA or NVS, rarely the app.

Ask the bench — it reads the chip directly ([FSD §6.7.3](../../../docs/Harness-FSD.md#673-chip-identity-via-post-apichipinfo)):

curl -X POST $TESTBENCH_URL/api/chip/info \\
  -H 'Content-Type: application/json' -d '{"slot": "SLOT3"}'
# -> "flash_size": "4MB", plus chip, revision, MAC and USB mode

It runs `esptool flash_id` on the Pi with the proxy stopped, so **it reboots the DUT** — a deliberate call, not something to poll. Stop any debug session first or it returns `409`.

Off the bench, the same thing over a local port:

esptool --port /dev/ttyACM0 flash_id     # "Detected flash size: ..."

/* Or from the firmware, the only way that survives a board swap */
uint32_t size = 0;
esp_flash_get_physical_size(NULL, &size);
ESP_LOGI(TAG, "flash %lu MB", (unsigned long)(size / (1024 * 1024)));

Then set `CONFIG_ESPTOOLPY_FLASHSIZE_<n>MB=y` in `sdkconfig.defaults` to the measured value and size the partition table to fit it. Setting `CONFIG_ESPTOOLPY_HEADER_FLASHSIZE_UPDATE=y` additionally lets esptool correct the header from the detected size at flash time — useful when one image serves boards that differ, but it does