/laya-integration

name: laya-integration
description: Add fast, local, typed decisions to any project with Laya, an open-source non-generative decision model (pip install laya). It classifies, routes, scores and answers yes/no questions about a piece of text, returning calibrated probabilities in roughly 20-35 ms on a laptop GPU, with no LLM call and no data leaving the machine. Use this skill whenever the user wants to classify or route text, triage tickets or emails, detect spam, phishing, toxicity or intent, put a guardrail in front of an agent or LLM, score something against a rubric, or replace an LLM-based classifier to cut latency or cost. Also use it when they mention Laya, Jev, a "System 1 model", "typed decisions" or a "decision model", even if they never name Laya.
metadata:
  author: brain function collapse
  source: https://brainfunctioncollapse.com/laya
  version: "1.1"

Integrating Laya

Laya answers typed questions about a piece of text. You give it a `state` (the text) and a dict of questions; it returns a probability for every option of every question in one forward pass. It never generates text, so it cannot answer off-schema, and it is small enough (322M-421M parameters) to run inside the user's own process.

Reach for it when the project needs a **decision**, not a sentence: which queue, is this spam, how urgent, should the agent stop. Do not reach for it when the task needs reasoning over several steps, arithmetic, extraction of free-form values, or any generated text. For those, keep the LLM and use Laya in front of it as a cheap first pass (see "Cascade" below).

Install and first call

pip install laya        # pulls torch, transformers, safetensors, huggingface_hub

import laya

agent = laya.load("convaiinnovations/laya")          # English checkpoint

result = agent.predict(
    {"subject": "Charged twice", "body": "Refund the duplicate today or we cancel."},
    {
        "department": {
            "type": "choice",
            "instructions": "Which department should handle this request?",
            "criteria": {
                "billing": "invoices, payments, refunds",
                "technical": "bugs, outages, system errors",
                "other": "everything else",
            },
        },
        "urgency": {
            "type": "score",
            "instructions": "How urgent is this request?",
            "criteria": ["not urgent", "soon", "critical deadline or blocking issue"],
        },
        "churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"},
    },
)
answers = result["answers"]

`state` may be a string, a dict or a list; dicts and lists are serialised to JSON. Load the model **once** at startup and reuse it. Loading takes 25-35 s; a call takes milliseconds.

The three question types

| type | `criteria` | what comes back in `answers[id]` | |---|---|---| | `choice` | dict `{option: description}` or a list of option names | `choice` (the winner), `probabilities` per option, `confidence` | | `score` | ordered list of level descriptions, lowest first | `score` (expected level, a float), `probabilities` per level, `legend`, `confidence` | | `noul` | optional `{"true": "...", "false": "..."}` | `noul` = P(true), `confidence` |

Every answer also carries `action.act_probability` from the model's act-or-escalate head, and `result["usage"]["input_tokens"]` reports the tokens consumed. `confidence` is one minus the normalised entropy of the distribution, so it is low whenever probability is spread out, even if the top option is right.

All questions in one `predict` call share a single forward pass, so ask everything you need about a state in one call rather than looping.

Writing questions that work

This matters more than anything else in the integration. Laya is an encoder doing something close to textual entailment, and it rewards questions shaped like that.

• **Ask what the text says, not what to do about it.** "Where is the bird relative to the gap?"

produced clean graded probabilities; "Which way must the bird move?" came out inverted on every checkpoint, because the option word "up" is pulled towards "above" in the state. Ask a perception question, then let your code map the answer to an action.

• **Put the state into words, never numbers.** Given "Bird altitude: 20. Gap altitude: 60." no

checkpoint could tell which was lower. If a decision depends on a comparison, threshold or sum, compute it in code and hand Laya the conclusion ("the bird is far below the gap").

• **Describe each option.** `{"billing": "invoices, payments, refunds"}` beats a bare

`["billing", ...]`, because the description is what the model matches the text against.

• **Keep option lists short.** Each option is truncated to 48 tokens and all options for a

question share a budget (192 tokens on the English checkpoint, 256 on the others). Upstream reports accuracy degrading past roughly 20 options. If a question raises `ValueError: ... options exceed head_max_len`, shorten the descriptions or split the question.

• **Keep the state short and front-loaded.** The English checkpoint reads 512 tokens in total,

the others 1024, and an over-long state is cut from the end. Put what matters first. For emails, `laya.email_state(subject, body, sender=...)` strips quoted replies and signatures.

• **Try two or three phrasings and measure.** Small wording changes move results a lot. One

concrete case: "blocked by a barrier" separated lanes far better than "blocked by a train".

Ready-made question sets exist for common jobs: `laya.triage_questions()`, `laya.email_questions()`, `laya.guard_questions()`, `laya.moderation_questions()`, `laya.router_questions()`. Use them as starting points and read what they contain before relying on them.

Choosing a checkpoint

agent    = laya.load("convaiinnovations/laya")                              # English, 421M
agent_ml = laya.load("convaiinn