# LLM & Model Configuration

> Applies to: ChengOS v0.1.0+ ｜ Last verified: 2026-08-12 ｜ Source: `deploy/config/providers.toml`, `crates/cheng-nodes/src/nodes/builtin/ai/llm`, `deploy/.env.example`

Model providers are configured once per installation and then selected per node. This page covers
the provider whitelist, what each provider can actually do, and how credentials attach.

## How model configuration is layered

Three layers, and it helps to know which one you are editing:

1. **`deploy/config/providers.toml`** — the provider *whitelist*. A provider that is not
   registered here cannot be used at all. It declares credential shape, capabilities, and default
   model lists.
2. **Credentials** — your keys and endpoints, stored encrypted in the vault, one entry per
   provider account.
3. **Node configuration** — which provider and model a given `ai/llm` or agent node uses,
   selected in the property panel.

## Registered providers

Sixteen providers ship registered. The capability columns are what the platform will actually
*use*, which is not always what the vendor's API supports:

| Provider | Chat | Embedding | Rerank | Native tool calling | Credential shape |
|---|:-:|:-:|:-:|:-:|---|
| OpenAI | ✅ | ✅ | — | ✅ | API key + optional endpoint |
| Anthropic (Claude) | ✅ | — | — | **—** | API key |
| Gemini | ✅ | ✅ | — | ✅ | API key + optional endpoint |
| DeepSeek | ✅ | — | — | ✅ | API key + optional endpoint |
| Moonshot | ✅ | — | — | ✅ | API key + optional endpoint |
| MiniMax | ✅ | — | — | ✅ | API key + optional endpoint |
| Zhipu | ✅ | — | — | ✅ | API key + optional endpoint |
| DashScope | ✅ | ✅ | ✅ | ✅ | API key + optional endpoint |
| Hunyuan | ✅ | — | — | ✅ | API key + optional endpoint |
| Qianfan | ✅ | — | — | ✅ | API key + optional endpoint |
| Ollama | ✅ | ✅ | — | **—** | Endpoint only |
| LM Studio | ✅ | ✅ | — | ✅ | Endpoint only |
| vLLM | ✅ | — | — | ✅ | Endpoint only |
| Cohere | — | ✅ | ✅ | — | API key |
| Jina | — | ✅ | ✅ | — | API key |
| Custom (OpenAI-compatible) | ✅ | ✅ | — | — | Provider name + endpoint + key |

**Two entries deserve attention before you build an agent.** Anthropic and Ollama do **not**
declare native tool calling — the current provider implementations do not encode tools or JSON
mode, so the capability is deliberately not advertised. If you are building a
[ReAct agent](/docs/agent-react-guide) with `use_function_calling = true`, choose a provider from
the ✅ column. Anthropic and Ollama remain excellent for plain `ai/llm` chat nodes; this is a
tool-calling limitation, not a quality judgment.

Cohere and Jina are embedding/rerank-only — they will not appear in a chat model dropdown, which
is correct rather than a bug.

## Protocol families

Providers are grouped by wire protocol, which is why so many are supported: most speak
OpenAI's chat-completions format.

| Family | Providers |
|---|---|
| `open_ai_chat_completions` | OpenAI, LM Studio, vLLM, DeepSeek, MiniMax, Zhipu, DashScope, Moonshot, Hunyuan, Qianfan |
| `anthropic_messages` | Anthropic |
| `gemini_generate_content` | Gemini |
| `ollama_chat` | Ollama |
| `custom_open_ai_compatible` | Custom |

## Adding a provider credential

In the admin console, add a credential for the provider. The form is generated from the
provider's declared `credential_schema`, so you are shown exactly the fields that provider needs:

| Schema | Fields |
|---|---|
| `api_key_only` | `api_key` (secret) |
| `api_key_with_endpoint` | `api_key` (secret) + optional `endpoint` |
| `endpoint_only` | `endpoint` — for self-hosted services |
| `custom_llm` | `provider_name`, endpoint, and key |

Secret fields are masked in the UI and redacted in API responses.

## Model lists

Providers with `fetch_models_enabled` retrieve their model list live from the API once a
credential is configured — so new models appear without a ChengOS update. When a live fetch is
unavailable, the `default_chat_models` and `default_embedding_models` declared in
`providers.toml` are used instead.

## Context windows and per-model overrides

Each provider declares a default context window, with pattern-matched overrides for models that
differ. OpenAI's default is 128K, but `gpt-3.5-turbo*` is overridden to 16385 — because a wrong
context window means either wasted capacity or requests that fail on length.

Overrides also declare output modalities for media models: `gpt-image-*` and `dall-e-*` output
images, `tts-*` outputs audio. That is what lets media generation route and validate correctly.

Anthropic's Claude 3.x/3.5 family declares 200K.

## Self-hosted and private endpoints

Ollama, LM Studio, and vLLM take an endpoint and no key. For a model server on a private address
you also need:

```bash
ALLOW_PRIVATE_LLM_ENDPOINTS=true
```

It defaults to `true`. Set it to `false` to block requests to private network ranges — worth doing
on an internet-facing installation, where a workflow pointed at an internal address becomes an
SSRF vector.

Anything else speaking the OpenAI format can use the `custom` provider.

## Selecting a model on a node

Fields using the `llm-model-select` control show one unified dropdown across every connected
provider. Node authors narrow it with `x-model-type`, `x-model-name-includes`,
`x-model-name-excludes`, and `x-provider-filter` — which is why an embedding field offers
embedding models rather than everything.

Model configuration on `ai/llm` and agent nodes can also be supplied as a JSON object through
`llm_config`, which is how a workflow parameterises the model instead of hardcoding it.

## Presets

`deploy/config/presets.yaml` defines named configuration patches for node types — a
`config_patch` shallow-merged into a target node's config. Presets declare a `preset_type`
category, a `node_type` they are valid for, a label, an optional `risk_level` (`low`, `medium`,
`high`), and `llm_visible`, which controls whether scoped LLM discovery may surface the preset.
`llm_visible` defaults to **false**, so a preset is invisible to models unless you opt in.

Per-workspace `presets.yaml` files override global presets with the same id (case-insensitive)
and may add new ones.

## Next steps

- [Environment variable reference](/docs/config-env-reference) — related settings.
- [Node configuration](/docs/editor-node-configuration) — the model selector control.
- [ReAct agent guide](/docs/agent-react-guide) — where tool-calling support matters.