# Media Input Nodes

> Applies to: ChengOS v0.2.0+ ｜ Last verified: 2026-09-25 ｜ Source: `crates/cheng-nodes/src/nodes/builtin/io`

Everything here converts non-text media into text a model can read. Pick by source type:
`io/document_to_text` for documents, `io/ocr` for images (its `mode` trades speed against accuracy,
from text-layer-only extraction up to the full engine), `io/speech_to_text` for audio. Video needs two
steps — `io/video_extract_audio` first, then speech-to-text; use `io/video_extract_frames` when the
information is visual rather than spoken. All of them take artifact ports, so wire
`io/file_upload` straight into them rather than passing content inline.

## Document To Text — `io/document_to_text`

Parse PDF, Word, Excel, TXT and other documents to text. Persists the full text as an artifact and returns a bounded preview.

### Inputs

| Field | Type | Required | Default | Description |
|---|---|:-:|---|---|
| `artifact` | object (port) | — | `null` | Document artifact reference (preferred). Connect from io/file_upload.document_artifact. Connection port, not a form field. |
| `file_path` | string | — | `null` | Local file path (legacy; mutually exclusive with artifact) |
| `base64_content` | string | — | `null` | Deprecated inline base64 content; prefer an artifact reference. Control: `textarea` |
| `filename` | string | — | `null` | Filename used to infer format (with base64_content) |
| `max_chars` | integer | — | `0` | Processing cap on extracted characters (0 = unlimited). Not the LLM preview budget. Control: `number` |
| `preview_bytes` | integer | — | `0` | Bounded UTF-8 preview budget for LLM (default 2 KiB). Full text is still persisted as an artifact. Control: `number` |
| `max_source_bytes` | integer | — | `0` | Maximum source bytes read before parsing (0 = 25 MiB default). Control: `number` |

### Outputs

| Field | Type | Description |
|---|---|---|
| `text_artifact` | object (port) | Artifact holding the complete extracted text. Connection port, not a form field. |
| `preview` | string | Bounded UTF-8 preview of the extracted text |
| `source_artifact` | object | Source document artifact, if any |
| `char_count` | integer | Characters in the full extracted text |
| `format` | string | Detected format (pdf/docx/xlsx/txt etc.) |
| `truncated` | boolean | Whether the preview is truncated vs full text |
| `warning` | string | 非致命告警（如工件持久化失败） |
| `text` | string | Deprecated; equals the bounded preview |

## OCR — `io/ocr`

Extract text from PDFs, images, and documents using local OCR. Persists full text, regions, layout, and structured results as artifacts; returns a bounded preview and metadata.

### Inputs

| Field | Type | Required | Default | Description |
|---|---|:-:|---|---|
| `artifact` | object (port) | — | `null` | Image/PDF artifact reference (preferred). Connect from io/file_upload.image_artifact or document_artifact. Connection port, not a form field. |
| `files_json` | array<object> (port) | — | `null` | Deprecated browser upload payload; prefer the artifact input. Connection port, not a form field. Control: `file-upload` |
| `file_path` | string | — | `null` | Deprecated local absolute file path on server; prefer the artifact input |
| `base64_content` | string | — | `null` | Deprecated inline base64 content; prefer the artifact input. Control: `textarea` |
| `filename` | string | — | `null` | Filename for format detection (used with base64_content) |
| `mime_type` | string | — | `null` | MIME type override (auto-detected from filename if not provided) |
| `mode` | enum | — | `auto` | Processing mode: Auto, TextPdfOnly, Light (Tesseract), Standard. Control: `select`. Options: `auto` (Automatically select the best engine based on input and hardware.), `text_pdf_only` (Only extract text from electronic PDFs; fail for scanned/image inputs.), `light` (Use lightweight OCR (Tesseract) even when heavier engines are available.), `standard` (Use the highest-quality engine available (PaddleOCR ONNX when enabled).) |
| `engine` | enum | — | `tesseract` | OCR engine for image/scanned PDF inputs: Tesseract by default, or PaddleOCR ONNX when installed. Control: `select`. Options: `tesseract` (Use the Tesseract CLI engine.), `paddle_onnx` (Use PaddleOCR via ONNX Runtime.) |
| `language` | enum | — | `auto` | Language hint: Auto, English, Simplified Chinese, Chinese+English. Control: `select`. Options: `auto` (Auto-detect language.), `eng` (English only.), `chi_sim` (Simplified Chinese only.), `chi_sim_eng` (Simplified Chinese + English (mixed).) |
| `max_chars` | integer | — | `0` | Processing cap on extracted characters (0 = unlimited). Not the LLM preview budget. Control: `number` |
| `preview_bytes` | integer | — | `0` | Bounded UTF-8 preview budget for LLM (default 2 KiB). Full text is still persisted as an artifact. Control: `number` |
| `min_confidence` | number | — | `0.0` | Minimum OCR confidence (0.0-1.0), 0 to skip confidence check. Control: `number` |
| `rules_json` | string | — | `null` | Optional JSON rules for extracting structured fields from OCR text. Control: `textarea` |
| `max_pages` | integer | — | `0` | Maximum pages to render for scanned PDFs (0 = default 5, max 50). Control: `number` |
| `render_dpi` | integer | — | `0` | DPI for rendering scanned PDF pages to images (0 = default 150). Control: `number` |

### Outputs

| Field | Type | Description |
|---|---|---|
| `preview` | string | Bounded UTF-8 preview of the recognized text |
| `text_artifact` | object (port) | Artifact holding the complete recognized text. Connection port, not a form field. |
| `regions_artifact` | object (port) | Artifact holding text regions with bounding boxes (JSON). Connection port, not a form field. |
| `layout_artifact` | object (port) | Artifact holding spatially-reconstructed reading-order text. Connection port, not a form field. |
| `structured_artifact` | object (port) | Artifact holding structured fields JSON when too large to inline. Connection port, not a form field. |
| `structured_json` | string | Small structured fields extracted using rules (kept inline; offloaded to structured_artifact when large) |
| `char_count` | integer | Number of characters extracted |
| `format` | string | Detected input format (pdf/image/text etc.) |
| `engine` | string | OCR engine used (none/pdf_text/tesseract/paddle_onnx) |
| `confidence` | number | OCR confidence score (0.0-1.0) |
| `status` | string | Result status: success/partial/failed/unsupported |
| `warning` | string | Warning about quality, completeness, or artifact persistence |
| `source_artifact` | object | Source image/document artifact, if any |
| `text` | string | Deprecated; equals the bounded preview |

## Speech To Text — `io/speech_to_text`

Transcribe audio files to text using OpenAI Whisper API, supports mp3/wav/m4a/ogg/flac/webm formats. Accepts an audio artifact reference (preferred) or local path. Persists the complete transcript as an artifact and returns a bounded preview.

### Inputs

| Field | Type | Required | Default | Description |
|---|---|:-:|---|---|
| `llm_config` | object (port) | ✅ | — | LLM config (connect to LLM config node for API Key, Endpoint and model). Connection port, not a form field. Control: `text` |
| `artifact` | object (port) | — | `null` | Audio artifact reference (preferred). Connect from io/file_upload.audio_artifact or io/video_extract_audio.audio_artifact. Connection port, not a form field. |
| `artifact_uri` | string | — | `null` | artifact://<id> URI (alternative to artifact; mutually exclusive) |
| `file_path` | string | — | `null` | Local absolute path of the audio file on server (legacy; mutually exclusive with artifact) |
| `base64_content` | string | — | `null` | Deprecated inline base64 content; prefer an artifact reference. Control: `textarea` |
| `filename` | string | — | `null` | Filename, required when using base64_content (e.g. audio.mp3) |
| `language` | string | — | `null` | Audio language (ISO-639-1, e.g. zh, en), leave empty for auto-detection |
| `prompt` | string | — | `null` | Prompt (optional), helps model recognize specific terms or speaking style. Control: `textarea` |
| `response_format` | enum | — | `text` | Output format: text (plain text) / json / verbose_json / srt / vtt. Control: `select`. Options: `text` (仅返回纯文本转录), `json` (返回 JSON（含 segments、duration 等）), `verbosejson` (返回带时间戳的详细 JSON), `srt` (SubRip 字幕格式), `vtt` (WebVTT 字幕格式) |
| `temperature` | number | — | `0.0` | Sampling temperature 0.0~1.0, 0 means most deterministic output |
| `preview_bytes` | integer | — | `0` | Bounded UTF-8 preview budget for LLM (default 2 KiB). Full transcript is still persisted as an artifact. Control: `number` |

### Outputs

| Field | Type | Description |
|---|---|---|
| `preview` | string | Bounded UTF-8 preview of the transcript; full text is in transcript_artifact |
| `transcript_artifact` | object (port) | Artifact holding the complete transcript / provider response body. Connection port, not a form field. |
| `detected_language` | string | Detected language code (only available in verbose_json format) |
| `duration_seconds` | number | Audio duration in seconds (only available in verbose_json format) |
| `char_count` | integer | Character count of the transcribed text |
| `response_format` | string | Echoed response format used by the provider |
| `source_artifact` | object | Source audio artifact reference, if any |
| `warning` | string | 非致命告警（如工件持久化失败） |
| `text` | string | Deprecated; equals the bounded preview |

## Video Extract Audio — `io/video_extract_audio`

Extract audio track from video file into a workspace audio artifact (via ffmpeg). Connect to Speech To Text node for transcription. Requires ffmpeg installed on system.

### Inputs

| Field | Type | Required | Default | Description |
|---|---|:-:|---|---|
| `artifact` | object (port) | — | `null` | Video artifact reference (preferred). Connect from io/file_upload.video_artifact. Connection port, not a form field. |
| `artifact_uri` | string | — | `null` | artifact://<id> URI (alternative to artifact) |
| `file_path` | string | — | `null` | Local absolute path of the video file (legacy; mutually exclusive with artifact) |
| `output_format` | enum | — | `wav` | Output audio format: wav (recommended) / mp3 / ogg / flac. Control: `select`. Options: `wav` (WAV（无损，与 Whisper 兼容性最好）), `mp3` (MP3（有损压缩，体积小）), `ogg` (OGG Vorbis), `flac` (FLAC（无损压缩）) |
| `sample_rate` | integer | — | `16000` | Sample rate (Hz), 16000 recommended for Whisper. Control: `number` |
| `channels` | integer | — | `1` | Audio channels: 1=mono (recommended), 2=stereo. Control: `number` |
| `start_seconds` | integer | — | `0` | Start extraction from this second (0 means from the beginning). Control: `number` |
| `duration_seconds` | integer | — | `0` | Duration of extracted audio (seconds), 0 means extract to the end. Control: `number` |

### Outputs

| Field | Type | Description |
|---|---|---|
| `audio_artifact` | object (port) | Extracted audio artifact reference; connect directly to io/speech_to_text.artifact. Connection port, not a form field. |
| `file_size` | integer | Size of the extracted audio file (bytes) |
| `format` | string | Output audio format (wav / mp3 / ogg / flac) |
| `ffmpeg_output` | string | Bounded ffmpeg stderr output (≤ 8 KiB), for debugging |
| `ffmpeg_output_truncated` | boolean | Whether ffmpeg stderr was truncated to fit the diagnostic budget |
| `source_artifact` | object | Source video artifact reference, if any |

## Video Extract Frames — `io/video_extract_frames`

Extract key frames from video at time intervals. Each frame is persisted as a JPEG artifact; node output stays compact (metadata + artifact references only). Downstream multimodal consumers materialize bytes on demand within configured budgets.

### Inputs

| Field | Type | Required | Default | Description |
|---|---|:-:|---|---|
| `artifact` | object (port) | — | `null` | Video artifact reference (preferred). Connect from io/file_upload.video_artifact. Connection port, not a form field. |
| `artifact_uri` | string | — | `null` | artifact://<id> URI (alternative to artifact) |
| `file_path` | string | — | `null` | Local absolute path of the video file (legacy; mutually exclusive with artifact) |
| `interval_seconds` | integer | — | `5` | Extract one frame every N seconds, default 5. Control: `number` |
| `max_frames` | integer | — | `20` | Maximum number of frames to extract, default 20. Control: `number` |
| `width` | integer | — | `768` | Output image width (pixels), 0 keeps original width. Control: `number` |
| `height` | integer | — | `0` | Output image height (pixels), 0 scales proportionally. Control: `number` |
| `jpeg_quality` | integer | — | `85` | JPEG compression quality 1-95, default 85. Control: `number` |
| `start_seconds` | integer | — | `0` | Start extracting from this second (0 means from the beginning). Control: `number` |
| `end_seconds` | integer | — | `0` | Stop extracting at this second (0 means to the end of video). Control: `number` |

### Outputs

| Field | Type | Description |
|---|---|---|
| `frames` | array&lt;object&gt; | Extracted frame list, each frame contains timestamp and a JPEG artifact reference (no inline bytes) |
| `frames_manifest` | object (port) | Artifact holding the JSON manifest of all extracted frames. Connection port, not a form field. |
| `first_frame_artifact` | object (port) | Convenience artifact reference for the first extracted frame. Connection port, not a form field. |
| `frame_count` | integer | Actual number of frames extracted |
| `video_duration_seconds` | number | Total video duration (seconds) |
| `source_artifact` | object | Source video artifact reference, if any |
| `status` | string | Result status: success (all frames extracted) / partial (some frames failed) / failed (no frames extracted) |
| `warning` | string | 非致命告警（如工件持久化失败、个别 ffmpeg 调用失败）。带总预算。 |
| `warning_truncated` | boolean | Whether the warning was truncated to fit the diagnostic budget |

## Next steps

- [Input & Output Nodes](/docs/nodes-input-output)
- [Media Generation Nodes](/docs/nodes-media-generation)
- [RAG Pipeline Design](/docs/data-rag-pipeline)