Storage, Redis & Vector Configuration

Applies to: ChengOS v0.1.0+ ｜ Last verified: 2026-08-12 ｜ Source: deploy/.env.example, crates/cheng-storage/src, crates/cheng-storage/src/janitor, crates/cheng-storage/src/artifact_store.rs, crates/cheng-vector/src

ChengOS has four stores, and they are not equally required:

PostgreSQL is the source of truth. The others accelerate or extend it.

PostgreSQL

DATABASE_URL=postgres://user:[email protected]:5432/master_router
DATABASE_MAX_CONNECTIONS=10
DATABASE_MIN_CONNECTIONS=1
DATABASE_CONNECTION_TIMEOUT=30

The hostname depends on your deployment mode, and this is the single most common misconfiguration:

Agent memory nodes need the connection split into parts as well as the URL, because they open their own connections:

DB_HOST=127.0.0.1
DB_PORT=5432
DB_NAME=master_router
DB_USER=tianai_db
DB_PASSWORD=…

DB_INSTALL_MODE picks how native installs provide the database:

Migrations run on startup when RUN_MIGRATIONS=true, or manually via Diesel — see Development Setup.

Redis

REDIS_URL=redis://:[email protected]:6379
REDIS_HOST=127.0.0.1
REDIS_PORT=6379
# REDIS_DB=0
REDIS_MAX_CONNECTIONS=10
REDIS_MIN_CONNECTIONS=1
REDIS_CONNECTION_TIMEOUT=30

ENABLE_REDIS=false        # native/hybrid installs

Redis carries the scheduler's warm queue and distributed execution lock, plus agent memory nodes and caching. Without it, scheduling and agent memory do not work; the rest of the system does.

As with PostgreSQL, agent memory nodes read REDIS_HOST / REDIS_PORT / REDIS_PASSWORD separately from REDIS_URL.

Valkey is a drop-in alternative.

Qdrant

QDRANT_URL=http://127.0.0.1:6334      # gRPC port, not 6333
ENABLE_QDRANT=false

Note the port: 6334 is Qdrant's gRPC endpoint, and that is what ChengOS connects to. Pointing at 6333 (the HTTP port) fails.

Deployment modes are local Docker, cloud, and self-hosted; cloud and authenticated self-hosted take an API key, and use_https plus a connection timeout (default 30 s) are configurable.

ChengOS uses two Qdrant features beyond plain vector search:

• Named vectors — one point holds parent_vec (384), child_vec (1024), and context_vec (256) with different dimensions. See RAG Pipeline Design.
• Built-in BM25 full-text, which is what makes the hybrid retrieval modes possible without a second search engine.

Collections are named kb_<workspace_id>_<kb_id>, so workspaces cannot collide — see Knowledge Bases.

The artifact store

Large files do not live in the database. The artifact store keeps immutable objects on disk:

<artifact_root>/
  objects/
    <artifact-id>/
      <safe-filename>
      metadata.json
  tmp/
    <write-in-progress files>

The write protocol is worth knowing because it explains the behaviour you observe:

1. Bytes go to a temporary file.
2. The file is hashed.
3. It is atomically renamed into objects/<id>/.
4. Only then does the database row become readable.

A failed registration removes the pending object. There is therefore no window in which a row points at bytes that do not exist, and a crash mid-upload leaves rubbish in tmp/ rather than a corrupt object.

Resolution is workspace-scoped and the resolved path is verified to lie inside the canonical artifact root, which defeats path traversal and symlink escape.

The root defaults to CHENG_WORKSPACE_ROOT/artifacts, falling back to ./workspaces/artifacts.

Per-workspace artifact settings — maximum file size, per-request total, LLM preview budget, retention — override the engine defaults. See Knowledge Bases & Workspace Context.

Data lifecycle: the janitor

A background janitor implements a three-phase strategy: soft delete (recoverable), cold backup (a future enhancement), then hard delete.

Defaults:

Plus orphaned-artifact cleanup, a batch size to avoid large transactions, a cleanup interval, and a dry-run mode that reports what it would delete without deleting.

The janitor uses a dedicated connection pool with a lower connection limit than the main pool, so cleanup can never starve business traffic, and it runs its blocking work on spawn_blocking so it does not tie up Tokio threads.

Run it in dry-run first on any deployment where the retention defaults might be too aggressive.

Secrets

CREDENTIAL_MASTER_KEY_1=<64 hex chars>   # openssl rand -hex 32
JWT_SECRET=<64 hex chars>
POSTGRES_PASSWORD=…                      # openssl rand -hex 16
REDIS_PASSWORD=…

CREDENTIAL_MASTER_KEY_1 is the AES-256-GCM key for the credential vault. Its numeric suffix is there for rotation. Lose it and every stored credential is unrecoverable — back it up separately from the database, because a database backup alone will not restore a working system.

Sizing

Start with the defaults. The engine's own concurrency is CPU cores × 3 because most nodes are I/O-bound, so database connections, not cores, are usually the first limit you meet.

Next steps

• Environment Variable Reference — every variable.
• Deployment Modes — which hostnames apply to you.
• RAG Pipeline Design — what actually uses Qdrant.