# Decision Providers & TypeSafe Jev

> Applies to: ChengOS v0.2.0+ ｜ Last verified: 2026-09-25 ｜ Source: `config/decision_providers.toml`, `config/decision_calibration.toml`, `crates/cheng-llm/src/decision_config.rs`, `crates/cheng-api/src/rest/handlers/decision/mod.rs`, `deploy/.env.example`

A **decision provider** answers one kind of question: *which of these declared options applies?*
It powers the [Decision Branch node](/docs/nodes-decision) and nothing else. This page covers
the three providers, how each is configured, and the calibration profiles that decide when an
answer is trusted. For building a router on top of them, see
[Semantic Routing](/docs/agent-semantic-routing).

## A decision provider is not a chat model

Decision providers are deliberately kept apart from the LLM providers in
[LLM & Model Configuration](/docs/config-llm-models):

| | Chat LLM provider | Decision provider |
|---|---|---|
| Answers | Free text, tool calls | One ID from a declared set, with a probability |
| Configured in | `providers.toml`, `/api/v1/llm/*` | `decision_providers.toml`, `/api/v1/decision/*` |
| Sees | History, tools, system prompt | Only the latest request, the route descriptions, the routing instruction, and (sticky mode) the previous route ID |
| Output checked | By the caller | Against the declared set before it can name a port |

So a Jev key never appears in a chat model picker, and a chat model is only ever a decision
provider through the explicit **LLM baseline** option below.

## The three providers

| Provider | Where it runs | Probabilities | Enabled by |
|---|---|---|---|
| **TypeSafe Jev** (`jev`, the default) | Hosted API at `https://api.typesafe.ai` | Calibrated | A user credential or the deployment key `JEV_API_KEY` |
| **Laya** (`laya`) | `laya serve` on infrastructure you operate | Calibrated only for a pinned model file | The operator: `LAYA_ENABLED=true` |
| **LLM baseline** (`llm`) | Any connected chat model | None — uncalibrated | Selecting a chat model on the node |

Jev is the default because it is the only provider with measured calibration out of the box.
The LLM baseline cannot use the calibrated threshold mode at all, because it reports no
probabilities; run it with threshold mode **None**.

## Jev credentials and the deployment key

A Jev connection is either the user's own credential or a key the operator manages for the
whole deployment.

**User credential** — type `jev` ("TypeSafe Jev") in the credential manager:

| Field | Required | Meaning |
|---|---|---|
| `api_key` | Yes | Sent as a Bearer token |
| `endpoint` | No | An approved HTTPS proxy or test endpoint; empty means `https://api.typesafe.ai` |
| `model` | No | Pinned model for everything using this credential |

The node's **Jev Credential** field selects one. Test a key before saving it with
`POST /api/v1/decision/credentials/test` (body: `provider`, `api_key`, optional `endpoint`, or
`credential_id` to re-test a stored key against an edited endpoint). The probe stores nothing
and is rate-limited per user.

**Deployment key** — leave the node's credential empty and set `JEV_API_KEY` in `deploy/.env`
(or in Settings → Deployment). Every decision branch without its own credential then uses it.

Either way, **the routed request text is sent to TypeSafe on every turn.** If that is not
acceptable for your data, use Laya.

## Models and pins

Calibration profiles are measured per **exact** model version, so a model name is resolved in a
fixed order:

1. The node's **Decision Model** field.
2. The credential's pinned `model`.
3. `JEV_DEFAULT_MODEL` (Jev only), read once at startup.
4. The shipped default in `decision_providers.toml`: `jev-1.13.0`.

An alias such as `jev-latest` is allowed, but the node gates on the profile of the model that
actually answered — so when the alias moves to a version without a profile, calibrated routing
falls back until you measure one. Move a pin together with a measured profile.

The model picker is fed by `GET /api/v1/decision/models` (a stored snapshot, no outbound call)
and `POST /api/v1/decision/models/refresh` (a live probe). A failed refresh never erases a
previous snapshot, and a snapshot never authorizes an execution: every call is validated again.

## Calibration profiles

A threshold means something only for the provider × model × language × task it was measured
on. `config/decision_calibration.toml` holds those measurements:

| Field | Meaning |
|---|---|
| `select_threshold` | Minimum probability of the selected route |
| `hold_follow_up_threshold` | Sticky mode: minimum probability that the request is a referential follow-up |
| `hold_previous_threshold` | Sticky mode: minimum probability of the previous route itself |
| `dataset` / `sample_size` / `measured_at` | Provenance of the measurement |

Shipped profiles (routing task, dataset `routing-eval-v2`):

| Profile | Select | Hold follow-up | Hold previous |
|---|---|---|---|
| `jev-1.13.0/en/routing-v2` | 0.70 | 0.75 | 0.30 |
| `jev-1.13.0/zh/routing-v2` | 0.70 | 0.75 | 0.30 |
| `laya-multilingual-f16/en/routing-v2` | 0.90 | never | never |
| `laya-multilingual-f16/zh/routing-v2` | 0.95 | never | never |
| `laya-english-q8_0/en/routing-v2` | 0.70 | never | never |

These numbers were measured against `jev-1.13.0` on 2026-09-23 and against the pinned Laya GGUF
files on 2026-09-24. They are not constants: **re-measure before relying on a profile for a new
route set, language, or model pin.** The labeled dataset is
`crates/cheng-llm/tests/fixtures/decision/routing_eval_set.json`, with recorded results beside it
and the Laya harness in `scripts/decision/laya_eval.py`, so an operator can reproduce a
measurement against their own routes.

A combination with no profile is **uncalibrated**: in calibrated mode the turn goes to the
fallback (or to the uncertain port, if the node separates it). No profile is ever inherited by
another provider or model.

To ship your own measurements, point `CHENG_DECISION_CALIBRATION_TOML` at a replacement file. An
invalid file is logged and ignored, leaving the shipped profiles in force.

## Laya: a local decision model

Laya is `laya serve` from ggmlc, run by the operator on the same host or a trusted network.
Users never see or set its address; it is deployment configuration, not a credential.

| Setting | Default | Meaning |
|---|---|---|
| `LAYA_ENABLED` | `false` | Master switch; invalid Laya settings keep it disabled |
| `LAYA_PORT` | `1215` | Port on the API host's loopback. Never 8080, which is `UI_PORT` |
| `LAYA_ENDPOINT` | — | Full address instead, e.g. `http://host.docker.internal:1215` from Docker |
| `LAYA_TRUSTED_NETWORK` | `false` | Required for a non-loopback plain-http endpoint; requests travel unencrypted |
| `LAYA_DEFAULT_MODEL` | `multilingual` | Family used when a node names none (`multilingual`, `english`, or an alias) |
| `LAYA_API_KEY` | — | Bearer token; `laya serve` reads the same variable |
| `LAYA_ARTIFACT_SHA256` | — | SHA-256 of the served GGUF; `LAYA_ARTIFACT_SHA256_<FAMILY>` pins one family |

**Pin the artifact.** `laya serve` reports only family names, so the SHA-256 pin is the only
identity a calibration profile can attach to. Without a pin — or with any other file, even a
re-quantization of the same checkpoint — Laya is uncalibrated. The shipped profiles match
`laya_multilingual_f16.gguf` and `laya_english_q8_0.gguf`; their digests are listed in
`deploy/.env.example`.

**Know its limits.** Laya takes at most 15 routes (16 options including the fallback). The
English family's 192-token head is small enough that eight routes with the default instruction
already exceed it; the editor reports the refusal. Laya is never sent the previous route, so it
asks no continuity question and cannot hold a route across turns. Its accuracy is well below
Jev's, which is why its thresholds are high.

**Running it.** In native mode, `hybrid/start.sh` launches it when `LAYA_BIN` and either
`LAYA_MODEL_PATH` (one GGUF) or `LAYA_MODELS_DIR` (both families) are set, verifies each GGUF
against its pinned digest, and sends a warm-up request; `LAYA_DEVICE` and `LAYA_LD_LIBRARY_PATH`
cover GPU builds. Leave them empty to run `laya serve` yourself. In Docker mode, run it on the
host and point `LAYA_ENDPOINT` at `host.docker.internal`.

`laya serve` has no `--host` option and listens on every interface, so firewall its port to the
API host or set `LAYA_API_KEY`. `chengos.sh doctor` fails a reachable Laya without a token. See
[Hardening a Public Deployment](/docs/deploy-hardening).

`GET /api/v1/decision/deployments/laya/status` reports readiness, and Settings → Decision
Models shows it.

## Other settings

| Variable | Meaning |
|---|---|
| `CHENG_DECISION_PROVIDERS_TOML` | Replacement for the shipped `decision_providers.toml`; an invalid file is ignored |
| `CHENG_DECISION_CALIBRATION_TOML` | Replacement for the shipped calibration profiles |

Both are read once at startup. Timeouts are TOML settings, not environment variables: Jev and
Laya default to 1500 ms (maximum 10 s); the LLM baseline adapts its deadline to each chat model
up to 30 s unless the node sets one. A missed deadline follows the node's failure mode and never
stalls the turn.

`GET /api/v1/decision/providers` returns the catalog the editor uses: capabilities, defaults,
calibration coverage, whether a deployment Jev key exists, and whether Laya is enabled.

## Next steps

- [Semantic Routing](/docs/agent-semantic-routing) — authoring routes and wiring the ports.
- [Decision Branch Node](/docs/nodes-decision) — every field of the node.
- [Environment Variable Reference](/docs/config-env-reference) — the rest of `deploy/.env`.