# The Jev Decision System

> Applies to: ChengOS v0.2.0+ ｜ Last verified: 2026-09-26 ｜ Source: `crates/cheng-llm/src/decision.rs`, `crates/cheng-llm/src/decision_service.rs`, `crates/cheng-nodes/src/nodes/builtin/ai/decision_branch.rs`, `config/decision_calibration.toml`, `crates/cheng-llm/tests/fixtures/decision/`

ChengOS can put a small, fast **decision model** in front of your agents. Before any large
language model runs, the decision model reads the user's latest request and picks one of the
routes you declared on the canvas — "email", "weather", "code" — or none of them. Each route
leads to an agent that carries only the tools it needs; everything unclear goes to the general
agent you already had.

This page explains why the system exists, how it is built, and what it measured. To build a
router, read [Semantic Routing with Decision Branch](/docs/agent-semantic-routing). For
credentials and providers, read [Decision Providers & TypeSafe Jev](/docs/decision-providers-and-jev).

## Why a decision system

One general agent with every tool attached works, but every turn pays for it:

- **Context.** The definitions of every tool are sent on every turn. With mail, web, and file
  tools attached that is about 4.3k–5.2k input tokens before the conversation even starts.
- **Accuracy.** More tools means more chances to pick the wrong one.
- **Privilege.** A request about the weather runs in an agent that could also send email.

The usual alternative, a single wrapper tool that lists the others on demand (Tool Hub), trades
context for an extra round trip — and in the measurements below it did not save context at all
on turns that actually needed a tool.

A decision model solves the problem one step earlier: it chooses the *path* before the chat
model sees anything. It never answers the user, never calls a tool, and can only choose among
routes the workflow author declared.

## How it works

Every routed turn passes two independent decision points:

```text
user message
  -> ai/decision_branch        choose a route (or the fallback) — decides context
  -> focused agent, or the general agent on fallback
  -> concrete tool call, if any
  -> approval policy           execute, ask the user, or deny — decides permission
```

The first decision only makes the context smaller. The second one — the deterministic approval
policy — is still the only thing that grants permission. A route can never unlock a tool call
that the approval policy would stop.

Inside ChengFlow the parts are:

| Part | Role |
|---|---|
| `ai/decision_branch` | The canvas node. You write route IDs and one-sentence descriptions; each route becomes a `route_<id>` port, plus a permanent `fallback_out`. |
| `DecisionService` (`cheng-llm`) | Builds a provider-neutral *Choice* question, enforces the deadline, validates the answer against the declared set, and records a text-free audit summary. |
| Decision providers | **TypeSafe Jev** (hosted, the default), **Laya** (runs on your own hardware), and the **LLM baseline** (any connected chat model, uncalibrated). |
| Calibration profiles | Measured thresholds per provider × model × language × task, in `config/decision_calibration.toml`. |

Decision providers are deliberately **not** chat providers: they never appear in a chat model
picker, and a chat model becomes a decision provider only through the explicit LLM-baseline
option.

## Design rules

These hold for every provider and every workflow:

- **The router is an optimization, not a gate.** Ordinary conversation, ambiguous or
  multi-intent requests, low confidence, timeouts, and provider outages all go to
  `fallback_out`. Wired to your existing agent, a misroute or outage degrades to today's
  behavior, never to a broken turn.
- **The answer is always one declared ID.** An out-of-set or malformed answer is rejected before
  it can name a port.
- **The conversation is delivered unchanged.** The chosen port carries the exact context the
  node received; history is never rewritten. Only a bounded summary is attached under
  `metadata["decision_route"]`.
- **The provider sees as little as possible.** Only the latest request, the route descriptions,
  the routing instruction, and — for follow-ups — the previous route ID. No system prompts, no
  tool definitions, no credentials.
- **Probabilities count only when calibrated.** A threshold is trusted only for the exact
  provider, model, and language it was measured on. Without a profile a turn falls back.
- **Edges follow IDs.** Reordering routes never moves an edge; unselected routes skip
  themselves downstream.

## What it measured

The go/no-go evaluation ran on 2026-09-23 against **routing-eval-v2**: 157 labeled requests (90
English, 67 Chinese) covering short and long single-intent requests, chit-chat, ambiguous and
multi-intent requests, referential follow-ups ("make it shorter"), new topics after a previous
route, and route lists of 3, 8, and 16. The decision model was `jev-1.13.0`.

The full interactive report — charts, per-category breakdown, threshold sweep — is published
here: **[Jev semantic routing evaluation](https://claude.ai/artifact/Rp4Hzx2KkkLkgpHHddygg7)**
(Chinese). Its key figures:

| | Result |
|---|---|
| Routing accuracy | **156 / 157** (99.4%; bootstrap 95% interval 98.1%–100%) |
| Answering-model context on tool-needing turns | **≈ 9× smaller** — 4,277 → 469 tokens (gpt-6-luna), 5,209 → 565 (GLM), median |
| Routing latency | **0.57 s** p50, 0.70 s p95 (chat-model routers: 1.4–2.9 s) |
| End-to-end completion | **On par** with the general agent (±2 of 101), 5–9 above Tool Hub |

**Routing decisions, all 157 requests.** A *wrong route* is the only error that loses
capability — the focused agent lacks the tool it needs. A *safe fallback* just sends the turn to
the general agent.

| Router | Correct | Safe fallback | Wrong route | p50 / p95 latency |
|---|---:|---:|---:|---|
| **Jev `jev-1.13.0`, calibrated** | **156** | 0 | 1 | 0.57 / 0.70 s |
| glm-4.5-air (uncalibrated) | 153 | 3 | 1 | 1.87 / 5.91 s |
| glm-5 (uncalibrated) | 152 | 3 | 2 | 1.92 / 5.24 s |
| gpt-4.1-mini (uncalibrated) | 146 | 5 | 6 | 2.91 / 3.66 s |
| gpt-6-luna (uncalibrated) | 137 | 20 | 0 | 1.39 / 1.83 s |

**End-to-end completion, 101 three-route requests** (email / weather / code, backed by the real
`mail`, `web`, and `file_ops_hub` tools). A tool-needing request counts when the right tool is
called; chit-chat and ambiguous requests count when no tool is called.

| Chat model | General agent | Tool Hub | Chat-model router | Jev router |
|---|---:|---:|---:|---:|
| gpt-6-luna | 92 | 85 | 92 | 90 |
| glm-4.5-air | 91 | 84 | 93 | 93 |
| glm-5 | 89 | 82 | 87 | 87 |

**Median context on the 63 tool-needing requests** (input tokens seen by the answering model,
all turns; Jev's own ~462 input tokens are billed by TypeSafe and enter no LLM context):

| Chat model | General agent | Tool Hub | After Jev routing | Reduction |
|---|---:|---:|---:|---:|
| gpt-6-luna | 4,277 | 4,723 | 469 | 9.1× |
| glm-4.5-air | 5,209 | 4,860 | 565 | 9.2× |
| glm-5 | 5,209 | 4,858 | 565 | 9.2× |

What the numbers say:

- **Jev made one mistake**, on the same multi-intent request every model got wrong (#098,
  "email the code review results to the team"). Multi-intent phrasing is the residual risk.
- **Chat models can route, but quality depends on the model** — and they report no
  probabilities, so no threshold can catch their unsure answers. They remain an explicit,
  uncalibrated option.
- **Tool Hub does not save context on tool-needing turns**: 59%–62% of them need an extra
  discovery turn whose result alone is ~4k tokens, and 14–19 of 63 tool-needing requests were
  answered without discovering any tool.
- **Calibration is stable.** Every select threshold from 0.60 to 0.75 gives 156/157; the shipped
  profile uses 0.70. Follow-ups are held on the previous route when the follow-up probability is
  ≥ 0.75 and the previous route's own probability is ≥ 0.30 — real follow-ups scored 0.79–1.00,
  new topics at most 0.04.

## Choosing a provider

| | TypeSafe Jev | Laya | LLM baseline |
|---|---|---|---|
| Runs | Hosted (`api.typesafe.ai`) | `laya serve` on your hardware | Any connected chat model |
| Request text leaves the deployment | **Yes, every routed turn** | No | Depends on the chat model |
| Calibrated probabilities | Yes | Only for a pinned model file | No — threshold mode *None* |
| Accuracy on routing-eval-v2 | 156/157 | 44–51 of 90 (en), 36 of 67 (zh), every wrong route multi-intent | Model-dependent (137–153/157) |
| Warm latency | 0.57 s p50 | 10–15 ms p95 on an RTX A4000 | 1.4–2.9 s p50 |
| Route limit | 16 | 15 (the fallback takes one option); fewer on the English model | 16 |

Jev is the default because it is the most accurate, the fastest hosted option, and ships with
calibration. Choose **Laya** when request text must stay on your own hosts: it is
conservative — it falls back far more often than Jev and never gets the previous route, so
sticky follow-ups do not apply — but it answers in milliseconds. A two-stage **cascade** (Laya
first, a local chat model only for its uncertain turns) recovers 7–9 correct routes per
scenario over Laya alone while calling the chat model on about 46% of turns. Neither local
option approaches Jev; both keep your data at home.

## Pairing routing with context compaction

Routing shrinks the *first* turn. Long runs grow through tool results instead, and a focused
agent can be tuned for that too. Starting with the release after v0.2.0, each
`agent/react_agent` takes an optional **compaction preset** — `standard` (today's behavior),
`aggressive`, or `lenient` — so the agent behind each route can compact the way its work needs:

```text
chat/input → ai/decision_branch ─route_email──► agent/react_agent  (standard)
                                ├route_code───► agent/react_agent  (lenient)
                                └route_data───► agent/react_agent  (aggressive)
```

The router itself is untouched; the preset is set on the agent, never inferred from a route
name. On the `describe → call` workload the aggressive preset's early archival of already-used
tool schemas cut cumulative request tokens by 34.8% in the offline corpus. It stays limited to
`aggressive` until it is measured on real replays.

## Limits and what's next

- The evaluation scored **first-turn tool selection**, not complete multi-step tasks, and the
  follow-up history was synthesized.
- Groups such as the 16-route cases (28 requests) are directional only. **Re-measure before
  changing routes, language, or model version** — a calibration profile applies only to what it
  was measured on.
- Route memory for follow-ups is kept in the API process for six hours. In a multi-instance
  deployment or after a restart, a follow-up may reach the general agent instead — safe, never a
  wrong route.
- Multi-intent requests go to the fallback by design. Serving them with more than one focused
  route is not part of the first release.
- Using a decision model to *raise* approval requirements for risky tool calls has been designed
  but deliberately deferred: the deterministic approval policy remains the only permission
  boundary.

## Next steps

- [Semantic Routing with Decision Branch](/docs/agent-semantic-routing) — build and tune a router.
- [Decision Providers & TypeSafe Jev](/docs/decision-providers-and-jev) — credentials, Laya, calibration profiles.
- [Decision Branch Node](/docs/nodes-decision) — every node field.
- [Tool-call Approval Policy](/docs/agent-approval-policy) — the permission boundary behind every route.