LLM & Model Configuration
Applies to: ChengOS v0.1.0+ ｜ Last verified: 2026-08-12 ｜ Source:
deploy/config/providers.toml,crates/cheng-nodes/src/nodes/builtin/ai/llm,deploy/.env.example
Model providers are configured once per installation and then selected per node. This page covers the provider whitelist, what each provider can actually do, and how credentials attach.
How model configuration is layered
Three layers, and it helps to know which one you are editing:
deploy/config/providers.toml— the provider whitelist. A provider that is not registered here cannot be used at all. It declares credential shape, capabilities, and default model lists.- Credentials — your keys and endpoints, stored encrypted in the vault, one entry per provider account.
- Node configuration — which provider and model a given
ai/llmor agent node uses, selected in the property panel.
Registered providers
Sixteen providers ship registered. The capability columns are what the platform will actually use, which is not always what the vendor's API supports:
| Provider | Chat | Embedding | Rerank | Native tool calling | Credential shape |
|---|---|---|---|---|---|
| OpenAI | ✅ | ✅ | — | ✅ | API key + optional endpoint |
| Anthropic (Claude) | ✅ | — | — | — | API key |
| Gemini | ✅ | ✅ | — | ✅ | API key + optional endpoint |
| DeepSeek | ✅ | — | — | ✅ | API key + optional endpoint |
| Moonshot | ✅ | — | — | ✅ | API key + optional endpoint |
| MiniMax | ✅ | — | — | ✅ | API key + optional endpoint |
| Zhipu | ✅ | — | — | ✅ | API key + optional endpoint |
| DashScope | ✅ | ✅ | ✅ | ✅ | API key + optional endpoint |
| Hunyuan | ✅ | — | — | ✅ | API key + optional endpoint |
| Qianfan | ✅ | — | — | ✅ | API key + optional endpoint |
| Ollama | ✅ | ✅ | — | — | Endpoint only |
| LM Studio | ✅ | ✅ | — | ✅ | Endpoint only |
| vLLM | ✅ | — | — | ✅ | Endpoint only |
| Cohere | — | ✅ | ✅ | — | API key |
| Jina | — | ✅ | ✅ | — | API key |
| Custom (OpenAI-compatible) | ✅ | ✅ | — | — | Provider name + endpoint + key |
Two entries deserve attention before you build an agent. Anthropic and Ollama do not
declare native tool calling — the current provider implementations do not encode tools or JSON
mode, so the capability is deliberately not advertised. If you are building a
ReAct agent with use_function_calling = true, choose a provider from
the ✅ column. Anthropic and Ollama remain excellent for plain ai/llm chat nodes; this is a
tool-calling limitation, not a quality judgment.
Cohere and Jina are embedding/rerank-only — they will not appear in a chat model dropdown, which is correct rather than a bug.
Protocol families
Providers are grouped by wire protocol, which is why so many are supported: most speak OpenAI's chat-completions format.
| Family | Providers |
|---|---|
open_ai_chat_completions | OpenAI, LM Studio, vLLM, DeepSeek, MiniMax, Zhipu, DashScope, Moonshot, Hunyuan, Qianfan |
anthropic_messages | Anthropic |
gemini_generate_content | Gemini |
ollama_chat | Ollama |
custom_open_ai_compatible | Custom |
Adding a provider credential
In the admin console, add a credential for the provider. The form is generated from the
provider's declared credential_schema, so you are shown exactly the fields that provider needs:
| Schema | Fields |
|---|---|
api_key_only | api_key (secret) |
api_key_with_endpoint | api_key (secret) + optional endpoint |
endpoint_only | endpoint — for self-hosted services |
custom_llm | provider_name, endpoint, and key |
Secret fields are masked in the UI and redacted in API responses.
Model lists
Providers with fetch_models_enabled retrieve their model list live from the API once a
credential is configured — so new models appear without a ChengOS update. When a live fetch is
unavailable, the default_chat_models and default_embedding_models declared in
providers.toml are used instead.
Context windows and per-model overrides
Each provider declares a default context window, with pattern-matched overrides for models that
differ. OpenAI's default is 128K, but gpt-3.5-turbo* is overridden to 16385 — because a wrong
context window means either wasted capacity or requests that fail on length.
Overrides also declare output modalities for media models: gpt-image-* and dall-e-* output
images, tts-* outputs audio. That is what lets media generation route and validate correctly.
Anthropic's Claude 3.x/3.5 family declares 200K.
Self-hosted and private endpoints
Ollama, LM Studio, and vLLM take an endpoint and no key. For a model server on a private address you also need:
ALLOW_PRIVATE_LLM_ENDPOINTS=true
It defaults to true. Set it to false to block requests to private network ranges — worth doing
on an internet-facing installation, where a workflow pointed at an internal address becomes an
SSRF vector.
Anything else speaking the OpenAI format can use the custom provider.
Selecting a model on a node
Fields using the llm-model-select control show one unified dropdown across every connected
provider. Node authors narrow it with x-model-type, x-model-name-includes,
x-model-name-excludes, and x-provider-filter — which is why an embedding field offers
embedding models rather than everything.
Model configuration on ai/llm and agent nodes can also be supplied as a JSON object through
llm_config, which is how a workflow parameterises the model instead of hardcoding it.
Presets
deploy/config/presets.yaml defines named configuration patches for node types — a
config_patch shallow-merged into a target node's config. Presets declare a preset_type
category, a node_type they are valid for, a label, an optional risk_level (low, medium,
high), and llm_visible, which controls whether scoped LLM discovery may surface the preset.
llm_visible defaults to false, so a preset is invisible to models unless you opt in.
Per-workspace presets.yaml files override global presets with the same id (case-insensitive)
and may add new ones.
Next steps
- Environment variable reference — related settings.
- Node configuration — the model selector control.
- ReAct agent guide — where tool-calling support matters.