# LLM Nodes

> Applies to: ChengOS v0.2.0+ ｜ Last verified: 2026-09-25 ｜ Source: `crates/cheng-nodes/src/nodes/builtin/ai`

`ai/llm` is the workhorse: one call, optional conversation context in and out, and a `tools` port that
turns it into a function-calling step. `ai/llm_config` exists so the model, provider, and parameters
live in one place and can be reused by every node downstream — configure once, connect many.
`ai/llm_branch` routes on fields of a structured response instead of on free text, which is far more
stable than parsing prose. `ai/context_merge` combines several AgentContext streams into the single
history a later call should see.

## AI Chat — `ai/llm`

Call large language model for dialogue generation

### Inputs

| Field | Type | Required | Default | Description |
|---|---|:-:|---|---|
| `context` | object (port) | — | `null` | AgentContext (optional, carries conversation history). Connection port, not a form field. |
| `llm_config` | object (port) | ✅ | — | LLM configuration (JSON string or object). Connection port, not a form field. Control: `text` |
| `instruction_prompt` | string | — | `""` | Instruction prompt (optional). Control: `textarea` |
| `user_message` | string | — | `""` | User message (can be empty if context.history already contains user messages). Control: `textarea` |
| `user_message_rich` | object (port) | — | `null` | Structured user message with multimodal parts. Connection port, not a form field. |
| `image_base64` | string (port) | — | `null` | Base64 image payload or data URL, connectable from io/file_upload.image_base64. Connection port, not a form field. |
| `tools` | array<any> | — | `null` | Available tools list (dynamically add ports) |

### Outputs

| Field | Type | Description |
|---|---|---|
| `context` | object (port) | AgentContext output port (contains conversation history). Connection port, not a form field. |
| `response` | string | Generated text response |
| `prompt_tokens` | integer | Tokens used in prompt |
| `completion_tokens` | integer | Tokens used in completion |
| `total_tokens` | integer | Total tokens used |
| `finish_reason` | string | Finish reason |
| `tool_calls` | array | Tool calls made by the LLM (for function calling) |
| `tool_results` | array | Tool execution results (when tools are connected and auto-executed) |
| `llm_config` | object (port) | LLM configuration used for this call (reusable by downstream nodes). Connection port, not a form field. Control: `text` |

## LLM Field Branch — `ai/llm_branch`

Route by field conditions: supports operator comparison, dot nested path, Schema hint generation, enhanced fallback metadata

### Inputs

| Field | Type | Required | Default | Description |
|---|---|:-:|---|---|
| `context` | object (port) | — | `null` | Input AgentContext (optional). Connection port, not a form field. Control: `code-editor` |
| `response` | string | — | `""` | LLM response text (JSON format). Control: `code-editor` |
| `custom_data` | string | — | `null` | Custom JSON data. Control: `code-editor` |
| `payload_source` | enum | — | `response` | Select data extraction source. Control: `select`. Options: `response` (从 response 字段解析 JSON), `context` (从完整 context 序列化为 JSON), `custom` (从 custom_data 字段取值) |
| `branch_fields` | array<object> | — | `[]` | Branch field list (click + to add; each entry auto-generates a corresponding output port) |
| `route_shortcut` | any | — | `null` | Route shortcut payload injected by ShortcutService (from execution metadata). Its `value` object is merged into the routing payload before field matching. |
| `strict_mode` | boolean | — | `false` | Error on JSON parse failure (false routes error reason to context_out fallback) |
| `generate_schema` | boolean | — | `false` | Enable auto-generated JSON Schema hint output to schema_hint port (can connect to upstream LLM's system_prompt) |

### Outputs

| Field | Type | Description |
|---|---|---|
| `branch_outputs` | array<any> (port) | Branch output list, corresponding 1:1 with branch_fields (AgentContext on match, null on miss). Connection port, not a form field. |
| `context_out` | object (port) | Fallback output: outputs AgentContext from this port when no branch fields match (includes routing diagnostic metadata). Connection port, not a form field. |
| `schema_hint` | object (port) | Auto-generated structured output hint (can connect to upstream LLM node's system_prompt port). Connection port, not a form field. |

## LLM Config — `ai/llm_config`

Select connected LLM model, output complete configuration

### Inputs

| Field | Type | Required | Default | Description |
|---|---|:-:|---|---|
| `model` | string | ✅ | — | Select a connected LLM model. Control: `llm-model-select` |
| `model_override` | string | — | `null` | Model override (format: credential_id::model_name, takes priority over model) |
| `temperature` | number | — | `0.699999988079071` | Controls randomness of generation, higher values are more random (0.0 - 2.0). Control: `slider` |
| `max_tokens` | integer | — | `5000` | Maximum number of tokens to generate (default 5000) |
| `top_p` | number | — | `1.0` | Nucleus sampling threshold (0.0 - 1.0). Control: `slider` |
| `frequency_penalty` | number | — | `0.0` | Reduce likelihood of repeating the same line (-2.0 to 2.0). Control: `slider` |
| `presence_penalty` | number | — | `0.0` | Encourage the model to discuss new topics (-2.0 to 2.0). Control: `slider` |
| `stream` | boolean | — | `true` | Enable real-time streaming of results |
| `ephemeral` | boolean | — | `false` | Run as a one-off call without loading or saving conversation history |
| `use_history` | boolean | — | `true` | Send prior conversation turns to the model; when off, each turn is answered on its own while the reply is still saved |
| `timeout_secs` | integer | — | `60` | API request timeout (1-600 seconds) |
| `json_mode` | boolean | — | `false` | Inject response_format: json_object (for weaker/open-source models, reduces non-JSON text; no system-level validation) |
| `compatibility_profile` | enum | — | `null` | Protocol template for custom OpenAI-compatible endpoints only (conservative \| openai \| deepseek \| minimax \| zhipu). Overrides the credential-level default; ignored for built-in providers. |

### Outputs

| Field | Type | Description |
|---|---|---|
| `llm_config` | object (port) | Complete LLM configuration. Connection port, not a form field. |

## Context Merge — `ai/context_merge`

Merge conversation contexts, intent text and data payloads into one AgentContext

### Inputs

| Field | Type | Required | Default | Description |
|---|---|:-:|---|---|
| `context_inputs` | array<any> | — | `[]` | 对话上下文端口组（context_input_0, context_input_1, ...） |
| `intent_inputs` | array<any> | — | `[]` | 意图文本端口组（intent_input_0, intent_input_1, ...） |
| `data_inputs` | array<any> | — | `[]` | 数据负载端口组（data_input_0, data_input_1, ...） |
| `intent_separator` | string | — | `null` | Separator placed between several intents (default: a blank line) |
| `custom_session_id` | string | — | `null` | Custom session_id (optional) |
| `system_message` | string | — | `null` | Additional system message (optional). Control: `textarea` |

### Outputs

| Field | Type | Description |
|---|---|---|
| `context` | object (port) | AgentContext output port (merged conversation). Connection port, not a form field. |
| `reply` | string | Assistant reply text from the merged context |
| `formatted_content` | string | Plain text of the merged conversation (human-readable; excludes attached data) |
| `data` | any | Payloads received on the data ports, verbatim, tagged by port |
| `stats` | object | Merge statistics |

## Next steps

- [Agent Nodes](/docs/nodes-agent)
- [LLM & Model Configuration](/docs/config-llm-models)
- [LLM Provider Errors](/docs/fix-llm-provider-errors)