/using-outbox

name: ring:using-outbox
description: "Using the transactional-outbox pattern across lib-streaming (writer) and lib-commons/v5/commons/outbox (repository + relay), in two modes. Sweep Mode detects DIY outbox tables, hand-rolled relay loops, send-and-pray emits, missing WithOutboxTx wrapping, and broker calls inside DB transactions. Reference Mode catalogs the writer/repository/envelope API and relay wiring. Go-only. Skip for non-Go or read-only services."

ring:using-outbox

When to use

Sweep mode:

• "Sweep for transactional outbox violations"
• "Find send-and-pray emits"
• "Are we wrapping DB transactions with WithOutboxTx?"
• "Migrate this service from DIY outbox to lib-streaming + lib-commons/outbox"
• "Audit relay loops for hand-rolled poller patterns"

Reference mode:

• "How does the transactional outbox pattern work?"
• "Which writer interface do I implement for X?"
• "What goes in OutboxEnvelope?"
• "How do I wire the relay loop?"
• "How does WithOutboxTx interact with MongoDB sessions?"

Skip when

• Working on non-Go services
• Service has no events to emit (pure read-side, BFF)
• Working on frontend code

Related

**Parent surface:** ring:using-lib-streaming (full streaming bus) **Repository side:** ring:using-lib-commons (lib-commons/outbox dispatcher, repository, handler registry) **Adjacent:** ring:instrumenting-streaming-events (eventable-point identification → emit wiring), ring:using-runtime (panic-safe relay loops), ring:using-assert (invariant checks on envelope decode)

---

The Pattern

The transactional outbox solves one operational invariant: **business state and the event that announces it must commit atomically, or not at all**. Without it, three failure modes are inevitable in production:

1. **Lost event.** Business state commits, the producer calls `broker.Emit`, the broker is down or the network blips — the event vanishes. The ledger now believes a transaction happened that no downstream consumer ever heard about. 2. **Phantom event.** Producer emits successfully, then the DB commit fails. Downstream consumers now act on a transaction that never happened. 3. **Send-and-pray.** Code paths that emit on a best-effort basis "and we'll log it if it fails" — a polite name for systematic data loss under sustained broker outages.

The outbox pattern fixes this by **writing the event to an `outbox` table inside the same database transaction as the business state**. Commit is atomic: either both rows persist or neither does. A separate process — the **relay** (also called dispatcher or poller) — reads pending outbox rows and publishes them to the broker, marking each row `PUBLISHED` on success. Delivery becomes at-least-once: if the relay crashes between publish and mark, the row stays pending and the next cycle retries. Consumers must be idempotent — that is the cost of at-least-once.

In lib-streaming the producer also uses the outbox as a **circuit-breaker fallback**. When a target's circuit is OPEN, `Emit` writes a route-aware `OutboxEnvelope` instead of attempting the broker call. When the breaker closes, the relay drains the backlog through the *originating target's adapter* — bypassing `Emit` itself, so replays cannot re-enter the circuit and cannot re-enqueue themselves. This is what `OutboxModeFallbackOnCircuitOpen` (the default) buys you: a broker outage degrades to a write-ahead log instead of dropped events.

Mode Selection

| Request Shape | Mode | |---|---| | "Sweep / audit / find DIY outbox / send-and-pray" | **Sweep** | | "How does the pattern work?" | **Reference** | | "Which interface do I implement?" | **Reference** | | "How do I wire WithOutboxTx in my repository layer?" | **Reference** | | "What is the OutboxEnvelope wire format?" | **Reference** |

---

SWEEP MODE

Dispatch 6 explorers in **one parallel batch**. Each writes its findings JSON; a synthesizer consolidates.

Phase 1: Outbox surface reconnaissance → outbox-surface.json
Phase 2: Multi-angle DIY sweep         → 6 × outbox-sweep-{N}-{angle}.json
Phase 3: Consolidated report           → outbox-sweep-report.md + outbox-sweep-tasks.json

Phase 1: Surface Reconnaissance

Before sweeping, determine what the service currently does:

1. Grep for `lib-streaming` and `lib-commons/v5/commons/outbox` imports in `go.mod` / source. 2. Locate broker-publish call sites (any of: `Emit`, `kafka.Produce`, `sqs.SendMessage`, `rabbitmq.Publish`, custom wrappers). 3. Locate DB-transaction boundaries (`db.BeginTx`, `*sql.Tx`, repository transactional helpers). 4. Emit `/tmp/outbox-surface.json`:

{
  "uses_lib_streaming": true,
  "uses_lib_commons_outbox": true,
  "broker_call_sites": [{"file": "...", "line": 0, "kind": "kafka|sqs|rabbitmq|custom"}],
  "tx_boundaries": [{"file": "...", "line": 0}],
  "has_outbox_table_migration": true,
  "has_relay_loop": false
}

If `uses_lib_streaming=false` AND `broker_call_sites` is non-empty → flag as high-risk send-and-pray candidate before angle dispatch.

Phase 2: 6-Angle DIY Sweep

⛔ STOP-CHECK BEFORE DISPATCH

Before emitting any Task call, count the explorers you intend to launch in this turn.

• Count MUST equal 6.
• If count < 6 → STOP. Do not partial-dispatch. Reconcile against the 6 angles below and try again.
• The 6 angles are the canonical sweep. No substitutions, no omissions.

⛔ MUST NOT trickle-dispatch

All 6 explorers leave in the SAME TURN, before reading any explorer output.

Forbidden sequences:

• Dispatch explorer 1 → read result → dispatch explorer 2
• Dispatch a subset → wait → dispatch the rest
• Dispatch follow-up explorers conditioned on partial output
• Loop sequentially over the angle list

If you find yourself about to dispatch an explorer in a turn AFTER any explorer has already returned a result → STOP. You violated parallel dispatch. Report the violation and mark the phase INCOMPLETE rather than completing the trickle.

Self-verify after dispatch

After the dispatch turn, verify a