/langgraph-decision-models

name: langgraph-decision-models
description: "INVOKE THIS SKILL when routing a LangGraph agent with a decision model (TypeSafe Jev, SemIf) instead of an LLM, or when auditing an existing agent for LLM calls that only produce a routing decision. Covers langchain-typesafe Noul/Choice/Score, reading answers correctly, threshold design, and LangSmith Gateway wiring."

<overview> A **decision model** answers typed questions about state and returns probabilities instead of prose. It replaces the common pattern of prompting an LLM, parsing its text, and branching on the result.

• **`Noul(instructions=...)`** — binary question, returns a probability of yes
• **`Choice(instructions=..., criteria={...})`** — picks one label, returns the full distribution plus confidence
• **`Score(instructions=..., criteria=[...])`** — grades against an ordered rubric, returns an expected value plus confidence

`TypeSafeClassifier` is a LangChain `Runnable[ClassifierRequest, ClassifierResponse]`, so it drops into a node like any other runnable. Up to 32 questions share one request and are answered independently — your code combines them.

**Reach for one when** a node generates text only so you can parse a decision out of it: routing, triage, filtering, guardrails, or per-item classification over a batch.

**Do not** reach for one when the node's output is the product (summaries, drafts, code) or when the judgment needs multi-step reasoning. A decision model classifies; it does not think. </overview>

---

Install and wire

`langchain-typesafe` is alpha (`0.0.1a3`) and `TypeSafeClassifier` is marked `@beta` — pin it and expect churn.

uv add langchain-typesafe

Three ways to reach a model. The classifier POSTs to `{base_url}/v1/systemone` with `Authorization: Bearer {api_key}`, so switching providers is constructor arguments only:

<ex-wiring> <python>

import os
from langchain_typesafe import TypeSafeClassifier

# 1. TypeSafe directly (Jev). Reads TYPESAFE_API_KEY when api_key is omitted.
classifier = TypeSafeClassifier(model="jev-latest")

# 2. SemIf, hosted on the LangSmith Gateway. Note: LangSmith key, not a TypeSafe key.
classifier = TypeSafeClassifier(
    model="semif-qwen3.5-4b",
    api_key=os.environ["LANGSMITH_API_KEY"],
    base_url="https://gateway.smith.langchain.com",
)

# 3. Jev through the Gateway (BYOK). The `typesafe/` prefix routes to a
# TYPESAFE_API_KEY stored in LangSmith workspace secrets. Without that secret
# every `typesafe/*` id returns 424 Failed Dependency -- before the model name is
# even validated, so a 424 does not confirm the id is real.
classifier = TypeSafeClassifier(
    model="typesafe/jev-1.13.0",
    api_key=os.environ["LANGSMITH_API_KEY"],
    base_url="https://gateway.smith.langchain.com",
)

</python> </ex-wiring>

---

The conversion pattern

Ask every question about a page/item in **one** request, put the typed response in state, and let a plain function route on it. The router is ordinary Python — testable without touching a network.

<ex-classify-and-route> <python>

from typing import TypedDict
from langchain_typesafe import ClassifierResponse, Noul, Score, TypeSafeClassifier
from langgraph.graph import StateGraph, START, END

QUESTIONS = {
    "relevant": Score(
        instructions="How relevant is this ticket to a billing problem?",
        criteria=["Unrelated.", "Possibly related.", "Directly about billing."],
    ),
    "angry": Noul(instructions="Is the customer expressing anger?"),
}

class State(TypedDict):
    text: str
    answers: ClassifierResponse
    route: str

classifier = TypeSafeClassifier(model="jev-latest")

def classify(state: State) -> dict:
    # One request, every question. They are answered independently.
    return {"answers": classifier.invoke(
        {"state": state["text"], "questions": QUESTIONS}
    )}

def route(state: State) -> str:
    a = state["answers"]
    if a.nouls["angry"].noul > 0.7:
        return "escalate"
    if a.scores["relevant"].score < 0.5:
        return "close"
    return "handle"

</python> </ex-classify-and-route>

Reading answers: `response.nouls[id].noul`, `response.choices[id].choice`, `response.scores[id].score`. Each view is keyed by your question id; `response.answers` holds them all.

---

Three traps when reading answers

These cause silent misrouting, not exceptions.

**1. `Score.score` is an expected value, not a level.** It is a probability-weighted average over the rubric and is routinely fractional. `score == 0` almost never fires — a "not responsive" item lands at `0.07`, not `0`. Always compare against a band.

if a.scores["relevant"].score < 0.5:   # correct
if a.scores["relevant"].score == 0:    # WRONG -- nearly never true

**2. Confidence measures distribution shape, not correctness.** On a `Score`, confidence reports how *concentrated* the rubric distribution is. An item sitting cleanly between two levels scores low confidence even when the model is entirely clear about it. A blanket `confidence < X -> escalate` rule therefore escalates items the model already decided. Gate on confidence only inside the ambiguous middle:

if score < NOT_RELEVANT:                          # decisive -- trust it
    return "close"
if score < RELEVANT or confidence < MIN_CONF:     # ambiguous -- escalate
    return "human_review"
return "handle"

**3. Thresholds do not transfer between models.** Calibration is part of the model. The same policy over the same items routes differently on Jev vs SemIf vs an LLM adapter. Re-tune thresholds whenever you change models, and pin the model id.

---

Question wording dominates accuracy

A loose question produces confident wrong answers, and no threshold fixes it. Use `criteria` to say what each outcome means, including what should *not* count.

In a measured case, "Is this a confidential communication with a lawyer?" scored a routine finance memo at **0.798**. Rewriting it to name the a