Decision Providers & TypeSafe Jev

Applies to: ChengOS v0.2.0+ ｜ Last verified: 2026-09-25 ｜ Source: config/decision_providers.toml, config/decision_calibration.toml, crates/cheng-llm/src/decision_config.rs, crates/cheng-api/src/rest/handlers/decision/mod.rs, deploy/.env.example

A decision provider answers one kind of question: which of these declared options applies? It powers the Decision Branch node and nothing else. This page covers the three providers, how each is configured, and the calibration profiles that decide when an answer is trusted. For building a router on top of them, see Semantic Routing.

A decision provider is not a chat model

Decision providers are deliberately kept apart from the LLM providers in LLM & Model Configuration:

So a Jev key never appears in a chat model picker, and a chat model is only ever a decision provider through the explicit LLM baseline option below.

The three providers

Jev is the default because it is the only provider with measured calibration out of the box. The LLM baseline cannot use the calibrated threshold mode at all, because it reports no probabilities; run it with threshold mode None.

Jev credentials and the deployment key

A Jev connection is either the user's own credential or a key the operator manages for the whole deployment.

User credential — type jev ("TypeSafe Jev") in the credential manager:

The node's Jev Credential field selects one. Test a key before saving it with POST /api/v1/decision/credentials/test (body: provider, api_key, optional endpoint, or credential_id to re-test a stored key against an edited endpoint). The probe stores nothing and is rate-limited per user.

Deployment key — leave the node's credential empty and set JEV_API_KEY in deploy/.env (or in Settings → Deployment). Every decision branch without its own credential then uses it.

Either way, the routed request text is sent to TypeSafe on every turn. If that is not acceptable for your data, use Laya.

Models and pins

Calibration profiles are measured per exact model version, so a model name is resolved in a fixed order:

1. The node's Decision Model field.
2. The credential's pinned model.
3. JEV_DEFAULT_MODEL (Jev only), read once at startup.
4. The shipped default in decision_providers.toml: jev-1.13.0.

An alias such as jev-latest is allowed, but the node gates on the profile of the model that actually answered — so when the alias moves to a version without a profile, calibrated routing falls back until you measure one. Move a pin together with a measured profile.

The model picker is fed by GET /api/v1/decision/models (a stored snapshot, no outbound call) and POST /api/v1/decision/models/refresh (a live probe). A failed refresh never erases a previous snapshot, and a snapshot never authorizes an execution: every call is validated again.

Calibration profiles

A threshold means something only for the provider × model × language × task it was measured on. config/decision_calibration.toml holds those measurements:

Shipped profiles (routing task, dataset routing-eval-v2):

These numbers were measured against jev-1.13.0 on 2026-09-23 and against the pinned Laya GGUF files on 2026-09-24. They are not constants: re-measure before relying on a profile for a new route set, language, or model pin. The labeled dataset is crates/cheng-llm/tests/fixtures/decision/routing_eval_set.json, with recorded results beside it and the Laya harness in scripts/decision/laya_eval.py, so an operator can reproduce a measurement against their own routes.

A combination with no profile is uncalibrated: in calibrated mode the turn goes to the fallback (or to the uncertain port, if the node separates it). No profile is ever inherited by another provider or model.

To ship your own measurements, point CHENG_DECISION_CALIBRATION_TOML at a replacement file. An invalid file is logged and ignored, leaving the shipped profiles in force.

Laya: a local decision model

Laya is laya serve from ggmlc, run by the operator on the same host or a trusted network. Users never see or set its address; it is deployment configuration, not a credential.

Pin the artifact. laya serve reports only family names, so the SHA-256 pin is the only identity a calibration profile can attach to. Without a pin — or with any other file, even a re-quantization of the same checkpoint — Laya is uncalibrated. The shipped profiles match laya_multilingual_f16.gguf and laya_english_q8_0.gguf; their digests are listed in deploy/.env.example.

Know its limits. Laya takes at most 15 routes (16 options including the fallback). The English family's 192-token head is small enough that eight routes with the default instruction already exceed it; the editor reports the refusal. Laya is never sent the previous route, so it asks no continuity question and cannot hold a route across turns. Its accuracy is well below Jev's, which is why its thresholds are high.

Running it. In native mode, hybrid/start.sh launches it when LAYA_BIN and either LAYA_MODEL_PATH (one GGUF) or LAYA_MODELS_DIR (both families) are set, verifies each GGUF against its pinned digest, and sends a warm-up request; LAYA_DEVICE and LAYA_LD_LIBRARY_PATH cover GPU builds. Leave them empty to run laya serve yourself. In Docker mode, run it on the host and point LAYA_ENDPOINT at host.docker.internal.

laya serve has no --host option and listens on every interface, so firewall its port to the API host or set LAYA_API_KEY. chengos.sh doctor fails a reachable Laya without a token. See Hardening a Public Deployment.

GET /api/v1/decision/deployments/laya/status reports readiness, and Settings → Decision Models shows it.

Other settings

Both are read once at startup. Timeouts are TOML settings, not environment variables: Jev and Laya default to 1500 ms (maximum 10 s); the LLM baseline adapts its deadline to each chat model up to 30 s unless the node sets one. A missed deadline follows the node's failure mode and never stalls the turn.

GET /api/v1/decision/providers returns the catalog the editor uses: capabilities, defaults, calibration coverage, whether a deployment Jev key exists, and whether Laya is enabled.

Next steps

• Semantic Routing — authoring routes and wiring the ports.
• Decision Branch Node — every field of the node.
• Environment Variable Reference — the rest of deploy/.env.