RAG Pipeline Design

Applies to: ChengOS v0.1.0+ ｜ Last verified: 2026-08-12 ｜ Source: crates/cheng-nodes/src/nodes/builtin/rag/constants.rs, crates/cheng-nodes/src/nodes/builtin/rag/nodes, crates/cheng-nodes/src/nodes/builtin/rag/services, crates/cheng-vector

RAG in ChengOS is five nodes and two pipelines. The indexing pipeline runs when documents change; the retrieval pipeline runs on every question. They share nothing at runtime except the knowledge base they read and write.

INDEXING                                  RETRIEVAL
 source text                               user query
     │                                         │
 rag/chunker      ──chunks_artifact──►     rag/retriever  ──documents_artifact──►
     │                                         │                    rag/formatter
 rag/document_indexer                          │                          │
     │                                         │                       ai/llm
     ▼                                         ▼
  knowledge base (Qdrant collection) ◄─────────┘

Configuration nodes

Three of the RAG nodes take config objects rather than loose fields, produced by dedicated nodes: ai/qdrant_config (where the vector store is), ai/embedding_config (which embedding model, and optionally a reranker), and ai/llm config for the steps that need a model.

Wiring config as an object rather than as twenty inputs is what lets one config node serve both the indexer and the retriever — and that matters, because the embedding model used to index must be the model used to retrieve. Sharing one config node makes the mismatch impossible.

Chunking

rag/chunker splits text into parent and child chunks. Four modes:

Sizes are measured in characters, not tokens, and that is a deliberate design decision: the chunker is a string splitter, not a tokenizer, and character counts are predictable, testable, and language-agnostic. Token counts are not — one CJK character is roughly 1–2 tokens while one English word is roughly 1.

The 512-character ceiling is a safety clamp applied at runtime: most embedding models cap at 512 tokens, and 512 characters leaves headroom in every language. A child chunk larger than that is clamped, not rejected.

Output is an artifact, not an array. The chunker returns chunks_artifact plus bounded previews (parent_preview, child_preview, preview_truncated) rather than the full chunk list inline. A large document produces thousands of chunks; passing them through node outputs would blow up execution records and any LLM context they touch. The inline parent_chunks / child_chunks fields still exist but are empty unless you set legacy_inline_chunks — treat them as deprecated. Symmetrically, the input accepts content_artifact / content_artifact_uri in preference to inline content.

Indexing

rag/document_indexer writes vectors into a knowledge base. Two modes:

Named vectors with different dimensions live in the same Qdrant point:

enhanced mode exists to feed the ultimate retrieval mode; if you are not going to retrieve in ultimate, standard costs less and indexes faster.

With generate_context: true (enhanced mode), an LLM writes a short contextual description for each chunk — the Contextual Retrieval technique — which is what context_vec embeds. This requires an llm_config and costs one model call per chunk, so it is off by default.

The indexer takes kb_id, not a collection name: the collection is resolved from the database and validated against the workspace, so a workflow cannot index into another workspace's collection by guessing a name. Prefer wiring chunks_artifact from the chunker; supplying content directly makes the indexer chunk internally with defaults.

Idempotency: pass version, and re-indexing an unchanged document returns skipped: true rather than rewriting every vector. This is what makes it safe to put the indexing workflow on a schedule.

Retrieval

rag/retriever has three modes, and the difference between them is the whole cost/quality dial of the system.

Economy runs a keyword search and a vector search and returns the union — cheap, and good enough when queries share vocabulary with the corpus.

Power first narrows the candidate set with the cheap parent_vec (pre-filter limit 50), then runs child_vec and BM25 over what survives, then fuses the two ranked lists with Reciprocal Rank Fusion:

RRF score = Σ  1 / (k + rank_i)        k = 60

RRF combines rankings, not scores, which is why it can merge a BM25 list and a cosine-similarity list without normalising incompatible score scales.

Ultimate adds HyDE (the LLM writes a hypothetical answer and embeds that instead of the question, which matches the corpus's phrasing better), uses the contextual descriptions written at index time, and reranks the top candidates (reranker top-k 10). It needs llm_config, an enhanced-indexed knowledge base, and a reranker in the embedding config — supported reranker models include Cohere rerank-multilingual-v3.0, Jina jina-reranker-v2-base-multilingual, and Qwen3 rerank at 0.6b / 4b / 8b.

Two other inputs matter:

• workspace_scope — ambient follows the runtime workspace; pinned locks this retriever to one configured workspace. Pin it when a shared workflow must always answer from one reference corpus regardless of who runs it.
• document_filter — a list of document ids to restrict the search to.

query accepts either a plain string or a chat context, so a retriever can be wired straight to a chat input node without an adapter in between.

Each result carries score, text, document_id, chunk_index, the context description if one exists, and retrieval_method — which method actually found it. That last field is the one to look at when tuning: it tells you whether BM25 or the vector search is doing the work.

Like the chunker, the retriever returns a documents_artifact and bounded previews.

Formatting the prompt

rag/formatter turns query plus retrieved documents into a prompt. Templates: qa (standard question answering), summary (document summarisation), or custom with placeholders {query}, {documents}, {count}. Toggle include_scores and include_sources to decide whether the model sees relevance scores and document ids — include sources when you want the answer to cite, omit them when you do not want the model narrating its retrieval.

Prefer wiring documents_artifact from the retriever; the inline documents input is a compatibility path.

Managing knowledge bases

rag/kb_manager handles the housekeeping: list (default), info, and delete. delete removes the collection — it is the one destructive node in the RAG set, so keep it out of agent-facing workflows.

A working pipeline

Indexing:
  io/read_file → rag/chunker → rag/document_indexer
                    ▲                  ▲
       ai/embedding_config ────────────┘
       ai/qdrant_config    ────────────┘
       workspace/knowledge_base (select) → kb_id

Answering:
  chat/input → rag/retriever → rag/formatter → ai/llm → chat/output

Start in economy + standard. Move to power when recall is the problem; move to ultimate + enhanced only when you have measured that reranking earns its latency and cost.

Next steps

• Knowledge Bases & Workspace Context — creating the target collection.
• Storage, Redis & Vector Configuration — running Qdrant.
• LLM & Model Configuration — embedding and generation models.