# LLM Provider Errors

> Applies to: ChengOS v0.1.0+ ｜ Last verified: 2026-08-13 ｜ Source: `deploy/config/providers.toml`, `crates/cheng-llm/src/error.rs`

Model failures fall into two groups that need opposite responses: **transient** ones worth
retrying, and **capability or configuration mismatches** that will fail identically forever. The
error type tells you which.

## Is it retryable?

| Retryable | Not retryable |
|---|---|
| `NetworkError` | `ProviderNotFound` |
| `Timeout` | `ConfigError` |
| `RateLimitExceeded` | `InvalidModel` |
| `ApiError` with status **5xx** | `ApiError` with other 4xx |
| `ApiError` with status **429** | `UnsupportedCapability` |
| `StreamError` (treated as network-related) | `UnsupportedInputModality` |
| | `InvalidProviderProfile` |
| | `RequestLimitExceeded` |
| | `ToolCallProtocolViolation` |
| | `ParseError`, `UnsupportedFeature` |

If you are retrying something in the right-hand column, you are burning quota on a request that
cannot succeed. Change the configuration instead.

## The most common failure: no native tool calling

An agent that produces no tool calls, or replies in prose describing what it *would* do, is
almost always this.

`agent/react_agent` defaults to `use_function_calling = true`, which requires the provider to
declare `native_tool_calling`. In the shipped `providers.toml`:

| Declares `native_tool_calling` | Does **not** declare it |
|---|---|
| OpenAI, Gemini, DeepSeek, LM Studio, vLLM | **Anthropic**, **Ollama** |

Anthropic's entry carries an explicit note: the current provider implementation does not encode
tools or JSON mode, so the capability is not declared. Ollama's implementation supports
`format: "json"` but not tools.

**Fix**: use a provider that declares the capability, or turn `use_function_calling` off and
accept the less reliable text-parsed tool protocol. See
[LLM & Model Configuration](/docs/config-llm-models).

## Capability errors, decoded

| Error | Means | Fix |
|---|---|---|
| `UnsupportedCapability` | The resolved profile lacks tools, JSON mode, or chat for this provider/model | Switch provider or model; do not retry |
| `UnsupportedInputModality` | You sent an image, audio, or file to a model whose profile does not declare it | Check `input_modalities` — Anthropic declares `["text", "image"]`, Ollama only `["text"]` |
| `RequestLimitExceeded` | Tool count, tool-name length, or input budget exceeded a declared limit; the message states `allowed` and `actual` | Reduce the request — fewer tools via [Tool Hub](/docs/agent-tools-and-hub), or less context |
| `InvalidProviderProfile` | The profile itself is inconsistent — bad pattern, contradictory capabilities, duplicate overrides — and it surfaces at **registry load time** | Fix `providers.toml`; this is not a runtime condition |
| `ToolCallProtocolViolation` | The model returned malformed argument JSON, an unregistered tool name, or ambiguous text calls | Non-retryable at the LLM layer; the agent policy decides whether to re-prompt |

`ToolCallProtocolViolation` is worth understanding: the LLM layer reports it as final, but the
**agent** may still re-prompt. So seeing one in a trace does not mean the turn failed.

## Provider configuration

Providers are a whitelist in `providers.toml`. Each entry declares its credential schema, what
it supports, and how it behaves.

Credential schemas:

| Schema | Fields |
|---|---|
| `api_key_only` | `api_key` (secret) |
| `api_key_with_endpoint` | `api_key` (secret), optional `endpoint` |
| `endpoint_only` | `endpoint` — for self-hosted services |
| `custom_llm` | `provider_name`, `api_key`, `endpoint`, optional `compatibility_profile` |

Sixteen providers ship: `openai`, `anthropic`, `ollama`, `lm-studio`, `vllm`, `gemini`,
`deepseek`, `minimax`, `zhipu`, `dashscope`, `moonshot`, `hunyuan`, `qianfan`, `cohere`, `jina`,
and `custom`.

`custom` takes a `compatibility_profile` — `conservative`, `openai`, `deepseek`, `minimax`, or
`zhipu`. Start with `conservative` for an unknown OpenAI-compatible endpoint: it assumes the
least and therefore fails least confusingly.

Behaviour is declared per provider: `protocol_family` (`open_ai_chat_completions`,
`anthropic_messages`, `ollama_chat`, `gemini_generate_content`, …), `tool_call_recovery`, and
flags such as Gemini's `force_non_streaming_with_tools`.

`model_overrides` narrow capabilities by pattern — for example `gpt-3.5-turbo*` is corrected to
its real 16K context rather than inheriting 128K, and Gemini's `imagen-*` and
`gemini-2.5-flash-image*` are marked as image-output models.

## Symptom table

| Symptom | Likely cause |
|---|---|
| `Provider not found: X` | Not in the `providers.toml` whitelist, or a typo in the node's provider field |
| `Configuration error: …` | Missing API key or endpoint; check the credential is bound and the schema's required fields are filled |
| `Invalid model: X is not supported by provider Y` | Model name wrong for that provider, or the dynamic model list failed and defaults are in use |
| `API error (status 401/403)` | Bad or expired key. Not retryable — replace the credential |
| `API error (status 429)` / `Rate limit exceeded` | Quota. Retryable; reduce concurrency ([subflow limits](/docs/editor-subflows)) |
| `API error (status 5xx)` | Provider outage. Retryable |
| `Request timed out` | Slow model or network; raise the node timeout before blaming the provider |
| `Parse error` | The endpoint is not returning the expected protocol — usually a `compatibility_profile` mismatch on a custom endpoint |
| Context-length errors | The model's real window is smaller than assumed; check for a `model_overrides` entry |
| Embeddings fail on a chat provider | `supports_embedding = false` — Anthropic, DeepSeek, vLLM among others |
| Reranking fails | Only Cohere and Jina declare `supports_reranker` |

## Self-hosted endpoints

Ollama, LM Studio, and vLLM use `endpoint_only` and are the usual local choices. Two checks
first:

1. **The endpoint must be reachable from the backend**, not from your browser. In Docker mode,
   `localhost` inside a container is the container.
2. **`fetch_models_enabled`** means the model list is pulled live; if the endpoint is
   unreachable the UI falls back to `default_chat_models`, and a model in that list may not
   actually be loaded.

For an agent on a local model, remember that Ollama does not declare native tool calling — LM
Studio and vLLM do.

## Diagnosing

1. Read the [execution trace](/docs/exec-traces-logs) — the error variant is in it.
2. Decide retryable versus not from the table above.
3. For capability errors, compare the node's provider and model against `providers.toml`.
4. For custom endpoints, try `compatibility_profile = "conservative"`.
5. Only then adjust prompts. A capability mismatch is never fixed by prompt wording.

## Next steps

- [LLM & Model Configuration](/docs/config-llm-models) — configuring providers properly.
- [Execution Failures](/docs/fix-execution-failures) — when the failure is not the model.
- [Your First ReAct Agent](/docs/quickstart-first-agent) — where the tool-calling trap bites first.