/agentsop-structured-output-picker

name: agentsop-structured-output-picker
version: 0.1.0
description: >-
  Decide where to enforce structured LM output (constrain at decode time with Outlines vs validate-and-retry with Instructor vs grammar with Guidance) and which failure stance to take (Assert/hard-fail vs Suggest/soft-retry). Use when an LM's output is parsed or typed by downstream code and you must pick one enforcement library plus its failure handling, when malformed output is burning tokens on retries, or when choosing between decode-time vs validation-time constraints for local vs API models.
domain: choosing and configuring a structured-output enforcement layer when LM output is consumed by code
overlay: true
enhances: [outlines, instructor, guidance]
source: |
  Local lib skills outlines / instructor / guidance (frontmatter + "When to Use");
  DSPy Assert/Suggest constraint primitives (dspy-sop-skill, arXiv 2312.13382);
  sibling Phase-D skill output-format-by-model (cross-link, not duplicate);
  provider docs for OpenAI structured outputs + Anthropic tool_use.
audience: |
  Coder-agents and pipeline authors wiring an LM whose output is parsed/typed by
  downstream code; anyone who has outlines / instructor / guidance installed and
  must pick one + decide failure handling.
status: tool-skill

Structured-Output-Picker — 在解码处约束，还是在校验处约束？

> **One-liner**: Three local libraries (Outlines, Instructor, Guidance) plus > provider-native structured outputs all "make the model emit valid structure", > but they enforce at *different points* and *fail differently*. Pick by **how > costly a malformed output is** and **whether you control the decoder**. Then > pick the failure stance — **Assert** (hard-fail + retry) vs **Suggest** (soft > nudge, degrade gracefully) — borrowed from DSPy's constraint primitives.

This is an **ENHANCE overlay**. The four enforcement mechanisms each have a working local skill; what no single one provides is the cross-library *which-one

• how-to-handle-failure* decision. That gap is hit every time an LM output is

consumed by code. For **what shape the content should take** (code vs JSON vs prose), descend first to `[[agentsop-output-format-by-model]]` — this skill assumes the shape is already chosen and asks only **how to enforce it**.

---

1. 何时激活 (When to activate)

Activate **after** you have decided the output *shape* (via `[[agentsop-output-format-by-model]]`) and the answer was "a typed/validated object", and **before** you write the parsing code.

| Trigger | Signal | |---|---| | LM output feeds a parser | `json.loads(resp)` / `Model.model_validate(...)` is in the next line of code | | You picked a typed shape | format-by-model said "JSON / Pydantic / typed field" — now: who enforces it? | | Repeated parse failures | `JSONDecodeError`, `ValidationError`, truncated/extra-prose responses in logs | | Library is already installed | `outlines`, `instructor`, or `guidance` is in the env and you must choose between them | | An enum / regex / range must hold | output must be one of N labels, a valid date, a bounded int | | You must decide failure stance | "if the model returns garbage, do I crash, retry, or accept-and-flag?" |

**Anti-triggers** (skip this skill):

• The content is **code / multi-step reasoning / long prose** — go back to

`[[agentsop-output-format-by-model]]`; enforcing a JSON grammar on code is the headline anti-pattern there (Aider 61%→20%). Enforcement strength is the wrong question when the *shape* is wrong.

• No code consumes the output yet (one-shot exploration).
• The shape is one token / one number — any parser works; no library needed.

---

2. 核心心智模型 (Core mental model)

2.1 The axis: where is the constraint applied?

   PROMPT ──────► DECODE ──────► RAW TEXT ──────► VALIDATE ──────► TYPED OBJECT
     │              │                                │
     │         constraint at DECODE             constraint at VALIDATE
     │         (grammar masks tokens)           (parse, check, retry on fail)
     │              │                                │
  ask nicely    Outlines / Guidance /          Instructor / DSPy Suggest /
  (weakest)     provider strict-mode           hand-rolled retry loop
                  ↑                                  ↑
            CANNOT emit invalid               CAN emit invalid, then
            structure — masked at             catches it and re-asks
            the logit level                   with the error injected

**The pick is governed by one question: how costly is a malformed output?**

• **A malformed output is cheap to recover from** (one extra API round-trip is

fine; the model is strong; failures are rare) → **constrain at validate** (Instructor-style retry). Simpler, model writes more naturally, no decoder access needed.

• **A malformed output is expensive or impossible to recover from** (no retry

budget, hard real-time, the output *must* be in a fixed enum/grammar, a single bad token corrupts a batch job) → **constrain at decode** (Outlines / Guidance grammar, or provider strict-mode). The model *cannot* emit invalid structure.

2.2 Two prerequisites that gate the choice

1. **Do you control the decoder?** Grammar/token-masking (Outlines, Guidance) requires logit access — i.e. **local/open weights** (Transformers, vLLM, llama.cpp) [[outlines]] [[guidance]]. Closed API models (GPT, Claude) expose only *their own* native structured-output / tool_use; you cannot bolt Outlines onto them. Instructor works *on top of* the API by parse-and-retry [[instructor]]. 2. **Is the content code-shaped?** If yes, stop — see §1 anti-triggers and `[[agentsop-output-format-by-model]]`. Grammar-constraining code yields *valid JSON containing degraded code*: enforcement cannot buy back content quality.

2.3 Failure stance is orthogonal — Assert vs Suggest

Independent of *which* library, you choose what happens on a constraint violation. DSPy names the two stances [dspy-so