/flash

name: flash
description: runpod-flash SDK and CLI for deploying AI workloads on Runpod serverless GPUs/CPUs.
user-invocable: true

Runpod Flash

Write code locally, iterate with `flash dev` — it runs your functions on remote Runpod GPUs/CPUs with hot-reload and live worker logs — then `flash deploy` to ship. `Endpoint` handles provisioning.

Setup

# install the CLI — requires Python 3.10-3.13
uv tool install runpod-flash
pip install runpod-flash

# auth option 1: browser-based login (saves token locally)
flash login
# headless: print URL instead of opening a browser
flash login --no-open
# max seconds to wait for browser auth (default 600)
flash login --timeout 300

# auth option 2: API key via environment variable
export RUNPOD_API_KEY=your_key

# scaffold a new project in ./my-project (writes AGENTS.md + CLAUDE.md)
flash init my-project
# scaffold in the current directory
flash init .
# overwrite existing files (-f)
flash init my-project --force
# update the CLI to the latest version
flash update
# pin a specific version (-V also works)
flash update --version 1.16.0

`flash init` writes `AGENTS.md` (+ a `CLAUDE.md` symlink). To add them to an existing project: `python -c "from runpod_flash.rules import install_agent_files; from pathlib import Path; install_agent_files(Path.cwd())"`.

CLI

`flash dev` is the canonical dev-server command (`flash run` still works as a hidden alias).

# local server at :8888, but functions run on REMOTE GPU/CPU workers;
# hot-reloads on save and streams the worker's logs live to your terminal
flash dev
# same, but pre-provision endpoints (no cold start on first call)
flash dev --auto-provision
# custom port/host; --reload/--no-reload toggles autoreload
flash dev --port 9000 --host 0.0.0.0
# build + deploy (auto-selects env if only one)
flash deploy
# build + deploy to "staging" environment
flash deploy --env staging
# deploy a specific app to an environment
flash deploy --app my-app --env prod
# build + launch local preview in Docker
flash deploy --preview
# build flags below also apply to deploy
flash deploy --no-deps --python-version 3.11
# list deployment environments
flash env list
# create "staging" environment
flash env create staging
# show environment details + resources
flash env get staging
# delete environment + tear down resources
flash env delete staging
# list flash apps in your account
flash app list
# create a flash app
flash app create my-app
# show an app's environments + builds
flash app get my-app
# delete an app and all its resources
flash app delete my-app
# list all active endpoints
flash undeploy list
# remove a specific endpoint
flash undeploy my-endpoint
# remove all endpoints (--interactive/-i to pick, --force/-f to skip prompts)
flash undeploy --all
# remove endpoints whose code no longer exists locally
flash undeploy --cleanup-stale

# build-only (no deploy) — mainly for debugging the artifact; `flash deploy` builds for you
# package the artifact without deploying (1500MB limit; torch auto-excluded)
flash build
# build flags: --no-deps, --exclude pkg1,pkg2, --output name.tar.gz, --python-version 3.11
flash build --no-deps

Dev vs Deploy

• `flash dev` — **iterate.** Local server at `:8888`, but your decorated functions

execute on **remote GPU/CPU workers**. Hot-reloads on save and **streams the worker's logs live** to the terminal. No build/upload/deploy wait — use this the whole time you develop.

• `flash deploy` — **ship.** Builds an artifact and deploys a stable endpoint. Slow

(build + upload + provision); only do this once the code works under `flash dev`.

`flash dev` ships **only the function body** to the worker, so a `NameError` for a module-level name surfaces immediately here. `flash deploy` imports the whole module and can mask that bug (see Gotcha #1). Develop against `flash dev` and you catch it first.

Autonomous Dev Loop

`flash dev` is a long-running server — run it in the background (don't block on it), capture its output, and drive it over HTTP. The captured log is the remote worker's live stream (cold start, model load, `print`s, tracebacks) — read it to debug.

flash dev > /tmp/flash-dev.log 2>&1 &                          # background; never run it blocking
until grep -q "flash dev  localhost:" /tmp/flash-dev.log; do sleep 2; done   # wait for startup
URL=$(grep -o "localhost:[0-9]*" /tmp/flash-dev.log | head -1)               # actual port (8888 bumps if taken)
curl -s "$URL/main/predict" -d '{"data": {...}}'               # dispatches to the remote worker

• **Read the real URL from the log** — flash auto-bumps the port if 8888 is in use, and

prints `✓ flash dev localhost:<port>` plus the route table.

• **Routes are namespaced by file**: `main.py`'s `/predict` is served at `/main/predict`.
• A handler typed `def predict(data: dict)` expects the arg as a top-level field — send

`{"data": {...}}`, not the bare object (otherwise 422).

• Edit a handler and save — hot-reload re-syncs the body; just re-send the request, no

redeploy. Add `--auto-provision` to skip the first-call cold start. `kill %1` when done.

Endpoint: Three Modes

Mode 1: Your Code (Queue-Based Decorator)

One function = one endpoint with its own workers.

from runpod_flash import Endpoint, GpuGroup

@Endpoint(name="my-worker", gpu=GpuGroup.AMPERE_80, workers=5, dependencies=["torch"])
async def compute(data):
    import torch  # MUST import inside function (cloudpickle)
    return {"sum": torch.tensor(data, device="cuda").sum().item()}

result = await compute([1, 2, 3])

Mode 2: Your Code (Load-Balanced Routes)

Multiple HTTP routes share one pool of workers.

from runpod_flash import Endpoint, GpuGroup

api = Endpoint(name="my-api", gpu=GpuGroup.ADA_24, workers=(1, 5), dependencies=["torch"])

@api.post("/predict")
async def predict(data: list[float]):
    import torch
    return {"result": torch.tensor(data, device="cuda").sum().ite