Media Input Nodes
Applies to: ChengOS v0.2.0+ ｜ Last verified: 2026-09-25 ｜ Source: crates/cheng-nodes/src/nodes/builtin/io
Everything here converts non-text media into text a model can read. Pick by source type:
io/document_to_text for documents, io/ocr for images (its mode trades speed against accuracy,
from text-layer-only extraction up to the full engine), io/speech_to_text for audio. Video needs two
steps — io/video_extract_audio first, then speech-to-text; use io/video_extract_frames when the
information is visual rather than spoken. All of them take artifact ports, so wire
io/file_upload straight into them rather than passing content inline.
Document To Text — io/document_to_text
Parse PDF, Word, Excel, TXT and other documents to text. Persists the full text as an artifact and returns a bounded preview.
Inputs
| Field | Type | Required | Default | Description |
|---|
artifact | object (port) | — | null | Document artifact reference (preferred). Connect from io/file_upload.document_artifact. Connection port, not a form field. |
file_path | string | — | null | Local file path (legacy; mutually exclusive with artifact) |
base64_content | string | — | null | Deprecated inline base64 content; prefer an artifact reference. Control: textarea |
filename | string | — | null | Filename used to infer format (with base64_content) |
max_chars | integer | — | 0 | Processing cap on extracted characters (0 = unlimited). Not the LLM preview budget. Control: number |
preview_bytes | integer | — | 0 | Bounded UTF-8 preview budget for LLM (default 2 KiB). Full text is still persisted as an artifact. Control: number |
max_source_bytes | integer | — | 0 | Maximum source bytes read before parsing (0 = 25 MiB default). Control: number |
Outputs
| Field | Type | Description |
|---|
text_artifact | object (port) | Artifact holding the complete extracted text. Connection port, not a form field. |
preview | string | Bounded UTF-8 preview of the extracted text |
source_artifact | object | Source document artifact, if any |
char_count | integer | Characters in the full extracted text |
format | string | Detected format (pdf/docx/xlsx/txt etc.) |
truncated | boolean | Whether the preview is truncated vs full text |
warning | string | 非致命告警（如工件持久化失败） |
text | string | Deprecated; equals the bounded preview |
OCR — io/ocr
Extract text from PDFs, images, and documents using local OCR. Persists full text, regions, layout, and structured results as artifacts; returns a bounded preview and metadata.
Inputs
| Field | Type | Required | Default | Description |
|---|
artifact | object (port) | — | null | Image/PDF artifact reference (preferred). Connect from io/file_upload.image_artifact or document_artifact. Connection port, not a form field. |
files_json | array<object> (port) | — | null | Deprecated browser upload payload; prefer the artifact input. Connection port, not a form field. Control: file-upload |
file_path | string | — | null | Deprecated local absolute file path on server; prefer the artifact input |
base64_content | string | — | null | Deprecated inline base64 content; prefer the artifact input. Control: textarea |
filename | string | — | null | Filename for format detection (used with base64_content) |
mime_type | string | — | null | MIME type override (auto-detected from filename if not provided) |
mode | enum | — | auto | Processing mode: Auto, TextPdfOnly, Light (Tesseract), Standard. Control: select. Options: auto (Automatically select the best engine based on input and hardware.), text_pdf_only (Only extract text from electronic PDFs; fail for scanned/image inputs.), light (Use lightweight OCR (Tesseract) even when heavier engines are available.), standard (Use the highest-quality engine available (PaddleOCR ONNX when enabled).) |
engine | enum | — | tesseract | OCR engine for image/scanned PDF inputs: Tesseract by default, or PaddleOCR ONNX when installed. Control: select. Options: tesseract (Use the Tesseract CLI engine.), paddle_onnx (Use PaddleOCR via ONNX Runtime.) |
language | enum | — | auto | Language hint: Auto, English, Simplified Chinese, Chinese+English. Control: select. Options: auto (Auto-detect language.), eng (English only.), chi_sim (Simplified Chinese only.), chi_sim_eng (Simplified Chinese + English (mixed).) |
max_chars | integer | — | 0 | Processing cap on extracted characters (0 = unlimited). Not the LLM preview budget. Control: number |
preview_bytes | integer | — | 0 | Bounded UTF-8 preview budget for LLM (default 2 KiB). Full text is still persisted as an artifact. Control: number |
min_confidence | number | — | 0.0 | Minimum OCR confidence (0.0-1.0), 0 to skip confidence check. Control: number |
rules_json | string | — | null | Optional JSON rules for extracting structured fields from OCR text. Control: textarea |
max_pages | integer | — | 0 | Maximum pages to render for scanned PDFs (0 = default 5, max 50). Control: number |
render_dpi | integer | — | 0 | DPI for rendering scanned PDF pages to images (0 = default 150). Control: number |
Outputs
| Field | Type | Description |
|---|
preview | string | Bounded UTF-8 preview of the recognized text |
text_artifact | object (port) | Artifact holding the complete recognized text. Connection port, not a form field. |
regions_artifact | object (port) | Artifact holding text regions with bounding boxes (JSON). Connection port, not a form field. |
layout_artifact | object (port) | Artifact holding spatially-reconstructed reading-order text. Connection port, not a form field. |
structured_artifact | object (port) | Artifact holding structured fields JSON when too large to inline. Connection port, not a form field. |
structured_json | string | Small structured fields extracted using rules (kept inline; offloaded to structured_artifact when large) |
char_count | integer | Number of characters extracted |
format | string | Detected input format (pdf/image/text etc.) |
engine | string | OCR engine used (none/pdf_text/tesseract/paddle_onnx) |
confidence | number | OCR confidence score (0.0-1.0) |
status | string | Result status: success/partial/failed/unsupported |
warning | string | Warning about quality, completeness, or artifact persistence |
source_artifact | object | Source image/document artifact, if any |
text | string | Deprecated; equals the bounded preview |
Speech To Text — io/speech_to_text
Transcribe audio files to text using OpenAI Whisper API, supports mp3/wav/m4a/ogg/flac/webm formats. Accepts an audio artifact reference (preferred) or local path. Persists the complete transcript as an artifact and returns a bounded preview.
Inputs
| Field | Type | Required | Default | Description |
|---|
llm_config | object (port) | ✅ | — | LLM config (connect to LLM config node for API Key, Endpoint and model). Connection port, not a form field. Control: text |
artifact | object (port) | — | null | Audio artifact reference (preferred). Connect from io/file_upload.audio_artifact or io/video_extract_audio.audio_artifact. Connection port, not a form field. |
artifact_uri | string | — | null | artifact://<id> URI (alternative to artifact; mutually exclusive) |
file_path | string | — | null | Local absolute path of the audio file on server (legacy; mutually exclusive with artifact) |
base64_content | string | — | null | Deprecated inline base64 content; prefer an artifact reference. Control: textarea |
filename | string | — | null | Filename, required when using base64_content (e.g. audio.mp3) |
language | string | — | null | Audio language (ISO-639-1, e.g. zh, en), leave empty for auto-detection |
prompt | string | — | null | Prompt (optional), helps model recognize specific terms or speaking style. Control: textarea |
response_format | enum | — | text | Output format: text (plain text) / json / verbose_json / srt / vtt. Control: select. Options: text (仅返回纯文本转录), json (返回 JSON（含 segments、duration 等）), verbosejson (返回带时间戳的详细 JSON), srt (SubRip 字幕格式), vtt (WebVTT 字幕格式) |
temperature | number | — | 0.0 | Sampling temperature 0.0~1.0, 0 means most deterministic output |
preview_bytes | integer | — | 0 | Bounded UTF-8 preview budget for LLM (default 2 KiB). Full transcript is still persisted as an artifact. Control: number |
Outputs
| Field | Type | Description |
|---|
preview | string | Bounded UTF-8 preview of the transcript; full text is in transcript_artifact |
transcript_artifact | object (port) | Artifact holding the complete transcript / provider response body. Connection port, not a form field. |
detected_language | string | Detected language code (only available in verbose_json format) |
duration_seconds | number | Audio duration in seconds (only available in verbose_json format) |
char_count | integer | Character count of the transcribed text |
response_format | string | Echoed response format used by the provider |
source_artifact | object | Source audio artifact reference, if any |
warning | string | 非致命告警（如工件持久化失败） |
text | string | Deprecated; equals the bounded preview |
Video Extract Audio — io/video_extract_audio
Extract audio track from video file into a workspace audio artifact (via ffmpeg). Connect to Speech To Text node for transcription. Requires ffmpeg installed on system.
Inputs
| Field | Type | Required | Default | Description |
|---|
artifact | object (port) | — | null | Video artifact reference (preferred). Connect from io/file_upload.video_artifact. Connection port, not a form field. |
artifact_uri | string | — | null | artifact://<id> URI (alternative to artifact) |
file_path | string | — | null | Local absolute path of the video file (legacy; mutually exclusive with artifact) |
output_format | enum | — | wav | Output audio format: wav (recommended) / mp3 / ogg / flac. Control: select. Options: wav (WAV（无损，与 Whisper 兼容性最好）), mp3 (MP3（有损压缩，体积小）), ogg (OGG Vorbis), flac (FLAC（无损压缩）) |
sample_rate | integer | — | 16000 | Sample rate (Hz), 16000 recommended for Whisper. Control: number |
channels | integer | — | 1 | Audio channels: 1=mono (recommended), 2=stereo. Control: number |
start_seconds | integer | — | 0 | Start extraction from this second (0 means from the beginning). Control: number |
duration_seconds | integer | — | 0 | Duration of extracted audio (seconds), 0 means extract to the end. Control: number |
Outputs
| Field | Type | Description |
|---|
audio_artifact | object (port) | Extracted audio artifact reference; connect directly to io/speech_to_text.artifact. Connection port, not a form field. |
file_size | integer | Size of the extracted audio file (bytes) |
format | string | Output audio format (wav / mp3 / ogg / flac) |
ffmpeg_output | string | Bounded ffmpeg stderr output (≤ 8 KiB), for debugging |
ffmpeg_output_truncated | boolean | Whether ffmpeg stderr was truncated to fit the diagnostic budget |
source_artifact | object | Source video artifact reference, if any |
Video Extract Frames — io/video_extract_frames
Extract key frames from video at time intervals. Each frame is persisted as a JPEG artifact; node output stays compact (metadata + artifact references only). Downstream multimodal consumers materialize bytes on demand within configured budgets.
Inputs
| Field | Type | Required | Default | Description |
|---|
artifact | object (port) | — | null | Video artifact reference (preferred). Connect from io/file_upload.video_artifact. Connection port, not a form field. |
artifact_uri | string | — | null | artifact://<id> URI (alternative to artifact) |
file_path | string | — | null | Local absolute path of the video file (legacy; mutually exclusive with artifact) |
interval_seconds | integer | — | 5 | Extract one frame every N seconds, default 5. Control: number |
max_frames | integer | — | 20 | Maximum number of frames to extract, default 20. Control: number |
width | integer | — | 768 | Output image width (pixels), 0 keeps original width. Control: number |
height | integer | — | 0 | Output image height (pixels), 0 scales proportionally. Control: number |
jpeg_quality | integer | — | 85 | JPEG compression quality 1-95, default 85. Control: number |
start_seconds | integer | — | 0 | Start extracting from this second (0 means from the beginning). Control: number |
end_seconds | integer | — | 0 | Stop extracting at this second (0 means to the end of video). Control: number |
Outputs
| Field | Type | Description |
|---|
frames | array<object> | Extracted frame list, each frame contains timestamp and a JPEG artifact reference (no inline bytes) |
frames_manifest | object (port) | Artifact holding the JSON manifest of all extracted frames. Connection port, not a form field. |
first_frame_artifact | object (port) | Convenience artifact reference for the first extracted frame. Connection port, not a form field. |
frame_count | integer | Actual number of frames extracted |
video_duration_seconds | number | Total video duration (seconds) |
source_artifact | object | Source video artifact reference, if any |
status | string | Result status: success (all frames extracted) / partial (some frames failed) / failed (no frames extracted) |
warning | string | 非致命告警（如工件持久化失败、个别 ffmpeg 调用失败）。带总预算。 |
warning_truncated | boolean | Whether the warning was truncated to fit the diagnostic budget |
Next steps