LLM Provider Errors

Applies to: ChengOS v0.1.0+ ｜ Last verified: 2026-08-13 ｜ Source: deploy/config/providers.toml, crates/cheng-llm/src/error.rs

Model failures fall into two groups that need opposite responses: transient ones worth retrying, and capability or configuration mismatches that will fail identically forever. The error type tells you which.

Is it retryable?

Retryable	Not retryable
`NetworkError`	`ProviderNotFound`
`Timeout`	`ConfigError`
`RateLimitExceeded`	`InvalidModel`
`ApiError` with status 5xx	`ApiError` with other 4xx
`ApiError` with status 429	`UnsupportedCapability`
`StreamError` (treated as network-related)	`UnsupportedInputModality`
	`InvalidProviderProfile`
	`RequestLimitExceeded`
	`ToolCallProtocolViolation`
	`ParseError`, `UnsupportedFeature`

If you are retrying something in the right-hand column, you are burning quota on a request that cannot succeed. Change the configuration instead.

The most common failure: no native tool calling

An agent that produces no tool calls, or replies in prose describing what it would do, is almost always this.

agent/react_agent defaults to use_function_calling = true, which requires the provider to declare native_tool_calling. In the shipped providers.toml:

Declares `native_tool_calling`	Does not declare it
OpenAI, Gemini, DeepSeek, LM Studio, vLLM	Anthropic, Ollama

Anthropic's entry carries an explicit note: the current provider implementation does not encode tools or JSON mode, so the capability is not declared. Ollama's implementation supports format: "json" but not tools.

Fix: use a provider that declares the capability, or turn use_function_calling off and accept the less reliable text-parsed tool protocol. See LLM & Model Configuration.

Capability errors, decoded

Error	Means	Fix
`UnsupportedCapability`	The resolved profile lacks tools, JSON mode, or chat for this provider/model	Switch provider or model; do not retry
`UnsupportedInputModality`	You sent an image, audio, or file to a model whose profile does not declare it	Check `input_modalities` — Anthropic declares `["text", "image"]`, Ollama only `["text"]`
`RequestLimitExceeded`	Tool count, tool-name length, or input budget exceeded a declared limit; the message states `allowed` and `actual`	Reduce the request — fewer tools via Tool Hub, or less context
`InvalidProviderProfile`	The profile itself is inconsistent — bad pattern, contradictory capabilities, duplicate overrides — and it surfaces at registry load time	Fix `providers.toml`; this is not a runtime condition
`ToolCallProtocolViolation`	The model returned malformed argument JSON, an unregistered tool name, or ambiguous text calls	Non-retryable at the LLM layer; the agent policy decides whether to re-prompt

ToolCallProtocolViolation is worth understanding: the LLM layer reports it as final, but the agent may still re-prompt. So seeing one in a trace does not mean the turn failed.

Provider configuration

Providers are a whitelist in providers.toml. Each entry declares its credential schema, what it supports, and how it behaves.

Credential schemas:

Schema	Fields
`api_key_only`	`api_key` (secret)
`api_key_with_endpoint`	`api_key` (secret), optional `endpoint`
`endpoint_only`	`endpoint` — for self-hosted services
`custom_llm`	`provider_name`, `api_key`, `endpoint`, optional `compatibility_profile`

Sixteen providers ship: openai, anthropic, ollama, lm-studio, vllm, gemini, deepseek, minimax, zhipu, dashscope, moonshot, hunyuan, qianfan, cohere, jina, and custom.

custom takes a compatibility_profile — conservative, openai, deepseek, minimax, or zhipu. Start with conservative for an unknown OpenAI-compatible endpoint: it assumes the least and therefore fails least confusingly.

Behaviour is declared per provider: protocol_family (open_ai_chat_completions, anthropic_messages, ollama_chat, gemini_generate_content, …), tool_call_recovery, and flags such as Gemini's force_non_streaming_with_tools.

model_overrides narrow capabilities by pattern — for example gpt-3.5-turbo* is corrected to its real 16K context rather than inheriting 128K, and Gemini's imagen-* and gemini-2.5-flash-image* are marked as image-output models.

Symptom table

Symptom	Likely cause
`Provider not found: X`	Not in the `providers.toml` whitelist, or a typo in the node's provider field
`Configuration error: …`	Missing API key or endpoint; check the credential is bound and the schema's required fields are filled
`Invalid model: X is not supported by provider Y`	Model name wrong for that provider, or the dynamic model list failed and defaults are in use
`API error (status 401/403)`	Bad or expired key. Not retryable — replace the credential
`API error (status 429)` / `Rate limit exceeded`	Quota. Retryable; reduce concurrency (subflow limits)
`API error (status 5xx)`	Provider outage. Retryable
`Request timed out`	Slow model or network; raise the node timeout before blaming the provider
`Parse error`	The endpoint is not returning the expected protocol — usually a `compatibility_profile` mismatch on a custom endpoint
Context-length errors	The model's real window is smaller than assumed; check for a `model_overrides` entry
Embeddings fail on a chat provider	`supports_embedding = false` — Anthropic, DeepSeek, vLLM among others
Reranking fails	Only Cohere and Jina declare `supports_reranker`

Self-hosted endpoints

Ollama, LM Studio, and vLLM use endpoint_only and are the usual local choices. Two checks first:

The endpoint must be reachable from the backend, not from your browser. In Docker mode, localhost inside a container is the container.
fetch_models_enabled means the model list is pulled live; if the endpoint is unreachable the UI falls back to default_chat_models, and a model in that list may not actually be loaded.

For an agent on a local model, remember that Ollama does not declare native tool calling — LM Studio and vLLM do.

Diagnosing

Read the execution trace — the error variant is in it.
Decide retryable versus not from the table above.
For capability errors, compare the node's provider and model against providers.toml.
For custom endpoints, try compatibility_profile = "conservative".
Only then adjust prompts. A capability mismatch is never fixed by prompt wording.

Next steps

LLM & Model Configuration — configuring providers properly.
Execution Failures — when the failure is not the model.
Your First ReAct Agent — where the tool-calling trap bites first.