Operations Runbook

Operations Runbook

1. Startup Checklist

1. kobi-ai points to engine with AGENT_SERVICE_URL.
2. BFF and engine share the same internal contract version (INTERNAL_API_CONTRACT_VERSION).
3. BFF service token and engine validation token are aligned (ENGINE_SERVICE_AUTH_TOKEN/INTERNAL_ENGINE_TOKEN).
4. Engine points to running MCP URL (MCP_SERVER_URL).
5. MCP has valid Supabase env names (runtime expects SUPABASE_SERVICE_KEY).
6. All services expose health endpoints and return success.

2. Health Checks

Engine

curl http://localhost:8000/health

MCP

curl -X POST http://localhost:3100/health -H "Content-Type: application/json" -d "{}"

BFF AI Route (requires user token)

curl -X POST http://localhost:3000/api/ai/chat \
  -H "Authorization: Bearer <USER_ACCESS_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"userMessage":"ping","threadId":"<thread_uuid>","agentType":"chat","projectId":"<project_uuid>","metadata":{}}'

BFF Canvas Read Routes (requires user token)

curl -G http://localhost:3000/api/canvas/workspace \
  -H "Authorization: Bearer <USER_ACCESS_TOKEN>" \
  --data-urlencode "agentId=strategy" \
  --data-urlencode "projectId=<project_uuid>" \
  --data-urlencode "threadId=<thread_uuid>" \
  --data-urlencode "includeEdges=true"

curl -G http://localhost:3000/api/canvas/context \
  -H "Authorization: Bearer <USER_ACCESS_TOKEN>" \
  --data-urlencode "agentId=strategy" \
  --data-urlencode "projectId=<project_uuid>" \
  --data-urlencode "threadId=<thread_uuid>" \
  --data-urlencode "maxNodes=50" \
  --data-urlencode "includeContent=true" \
  --data-urlencode "includeRelationships=true"

For full examples including internal route headers, see API and Event Contracts.

3. Critical Runtime Verifications

1. First message path materializes thread via /api/chat/threads before /api/ai/*.
2. /api/ai/* calls include normalized metadata only.
3. BFF forwards internal headers (X-Contract-Version, X-User-Id, X-Project-Id, X-Thread-Id, X-Agent-Type).
4. Engine accepts request and logs workflow execution.
5. MCP receives tool calls with approved context fields for side-effect tools.
6. No legacy auto-reply fallback path is executed in chat runtime.

4. 400 and 403 Triage Order

1. BFF schema/allowlist errors:

• Unknown request keys.
• Unknown metadata keys.
• threadId/sessionId mismatch.
• missing thread materialization (threadId missing or new).

2. Engine internal contract errors:

• Missing/invalid X-Contract-Version.
• Missing required trusted headers.
• Header/body claim mismatch.

3. Scope ownership errors:

• Thread not found for user.
• Project and thread mismatch.

4. MCP/tool errors:

• Missing approved/scope IDs for mutation tools.
• Tool not registered in MCP.

5. Smoke Test Matrix

1. Non-streaming chat success path.
2. Streaming chat emits done.
3. HITL interrupt and approve resume path.
4. HITL interrupt and reject resume path.
5. Canvas workspace/context reads with scoped query.
6. Canvas mutation with idempotency key.
7. One MCP mutating tool call verifies persistence and idempotent retry.

6. Known Current Gaps

• Engine configs still include some non-registered interaction tool names.
• propose_deep_research MCP registration currently lacks shared auth+gate set wiring in tool registry lists.
• Internal canvas pull endpoint is ready in BFF, but current engine path does not call it.