Channel Problems
Applies to: ChengOS v0.1.0+ | Last verified: 2026-08-13 | Source:
crates/cheng-adapters,crates/cheng-api/src/middleware/channel_auth.rs,crates/cheng-api/src/services/adapter_health.rs,crates/cheng-core/src/domain/channel
Channel problems split cleanly by where the message stops, so start by locating the stop point rather than guessing:
platform → webhook/worker → auth → normalize → conversation → workflow → reply → platform
① ② ③ ④ ⑤ ⑥ ⑦
| Stops at | Symptom |
|---|---|
| ① Platform never delivers | Nothing in the logs at all |
| ② Transport | Nothing arrives, but the channel is active |
| ③ Auth | 401 responses |
| ④ Normalisation | Delivery logged, no message stored |
| ⑤ Conversation | Message stored, no run |
| ⑥ Workflow | Run starts and fails |
| ⑦ Reply | Run succeeds, nothing sent back |
Connection states
unconfigured → configuring → active
│
health check fails → degraded → (auto-reconnect) → active
│
disconnected (manual) error { message } (needs manual intervention)
| State | Meaning |
|---|---|
unconfigured | No credentials or config yet |
configuring | Setup in progress — webhook registering, OAuth redirect, QR scan waiting |
active | Connected and operational |
degraded | Health check failed or token expired; auto-recovery in progress |
disconnected | Manually disconnected or revoked |
error | Unrecoverable; needs manual intervention |
A channel stuck in configuring almost always means a multi-step flow was never completed —
the OAuth redirect was not followed, or the QR was not scanned. Check setup_data.
Nothing arrives
First: is it a worker-based platform? Telegram polling, Slack Socket Mode, and DingTalk
Stream depend on a background worker. The health service treats the manager's is_running() as
the authoritative liveness signal precisely because a valid credential does not mean messages
are arriving if the worker has exited.
If the worker died, the channel goes degraded within one health interval
(ADAPTER_HEALTH_INTERVAL_SECS, default 300 s). Recovery calls the manager's restart() —
connect() alone only validates credentials and produces a false recovery where the database
says active while nothing is received. If you see active and silence, suspect exactly this.
For webhook platforms (WhatsApp, WeCom, Slack in webhook mode):
| Check | Why |
|---|---|
PUBLIC_BASE_URL is your real public HTTPS domain | Webhook URLs are computed from it at runtime |
| The URL is publicly reachable and not localhost | Telegram webhook mode validates both and fails connect with an explicit message |
| The platform console shows the webhook registered | Some platforms silently drop a failed registration |
| The verification challenge succeeded | WhatsApp sends a one-time GET with hub.* params; Slack posts url_verification |
Running behind NAT or on a laptop? Choose a transport that needs no inbound connection: Telegram polling (the default), Slack Socket Mode, or DingTalk Stream. See Messaging Channels.
Authentication failures
Channel keys are chk_{credential_id_hex}_{secret_hex} — the chk_ prefix, 32 hex characters
of credential id, and 32 hex characters of secret. They are stored as credentials with
provider = "channel_adapter".
Error codes, all returning 401:
| Code | Means |
|---|---|
MISSING_TOKEN | No Authorization header |
INVALID_KEY_FORMAT | Wrong prefix, wrong length, or a malformed credential id |
KEY_NOT_FOUND | The credential id does not exist |
WRONG_PROVIDER | The credential exists but is not a channel_adapter credential |
SECRET_MISMATCH | The secret half does not verify |
INVALID_KEY | Generic key rejection |
Anything else is a 500 — a server problem, not your key.
A key may also be scoped: to one app_id (platform) and to one exact channel_id. A key
that works for one channel and 401s on another is scoped, not broken.
Browser frontends must never use chk_ keys. They use JWT only; adapter keys belong to
server-side platform services. See Channel Routing.
Note that the /webhooks/:channel_uuid endpoints are deliberately unauthenticated —
authenticity comes from the adapter's signature check, not a bearer token. A 401 there means
something else is wrong.
Signature and challenge failures
Each platform verifies differently, and a Normalization or auth error usually points at the
wrong secret:
| Platform | Mechanism | Common cause |
|---|---|---|
| Telegram | X-Telegram-Bot-Api-Secret-Token equality | webhook_secret mismatch between config and registration |
HMAC-SHA256 on X-Hub-Signature-256 | Wrong signing_secret (the app secret, not the access token) | |
| Slack | HMAC-SHA256 on X-Slack-Signature with timestamp | Wrong signing_secret, or host clock skew |
| WeCom | SHA1 signature + AES-CBC encrypted XML | Wrong token or encoding_aes_key (must be 43 base64url chars) |
Message arrives, nothing runs
| Check | Detail |
|---|---|
| Is a workflow bound? | bound_workflow_id on the channel record. The workflow is never inferred from channelId |
Is the channel enabled? | |
| Was the message actionable? | Empty text-type messages are silently skipped; delivery status updates normalise to an empty list |
| Was it a command? | A message starting with ~ is a channel command, consumed instead of forwarded |
| Is the workflow itself valid? | See Execution Failures |
Duplicates and lost context
Duplicate replies usually mean deduplication is not working. Inbound messages carry
external_message_id for exactly this; a platform redelivering is normal and harmless when the
id is stable.
Conversation resolution is idempotent on (workspace_id, app_id, external_chat_id), so
repeated calls return the same conversation. If a user keeps landing in a fresh conversation,
external_chat_id is not stable — check what the adapter is putting there for group versus
direct chats.
Conversation rotation (~new, or a workflow switch) retires the old conversation by
suffixing its external_chat_id with __retired_<ts> in the same transaction that creates the
new one. Seeing a retired suffix in the database is normal, not corruption.
Reply not delivered
The outbound command is one of send_text, send_media, reply, react, or typing. Check
the adapter's declared capabilities: Telegram supports typing indicators but not reactions or
edits; Slack supports reactions, edits, and deletes but not typing. Attempting an unsupported
action returns NotSupported.
Error categories
Adapter errors are stable semantic categories, and the category tells you where to look:
| Category | Look at |
|---|---|
Config | Missing or wrong connection_config field |
Auth | Invalid token, expired credential, permission denied |
PlatformApi | The platform returned an error — often quota or a revoked app |
Normalization | Payload shape unexpected; check raw in the stored message |
NotSupported | The operation is not available on this adapter |
Internal | A ChengOS bug — file it with the trace |
Diagnostic checklist
1. GET /workspaces/:ws/channels/:id/status → connection state
2. GET /workspaces/:ws/channels/:id/capabilities → is the action even supported
3. Check the health interval has elapsed since the last change
4. For worker platforms, confirm the worker is running, not just the credential valid
5. Check bound_workflow_id
6. Check the conversation for a stored message → distinguishes ④ from ⑤
7. Check the execution trace → distinguishes ⑥ from ⑦
Next steps
- Messaging Channels — adapters and their configuration.
- Channel Routing & Session Mapping — bindings and commands.
- Authentication — the
chk_key scheme in full.