Architecture

The piecesCopy Anchor Link

browser ──► kai (Node) ─┬─► Postgres   data, under row-level security
  │          API + SPA  ├─► Redis      run event streams, rate limits, caches
  │                     ├─► S3         attachments, large tool outputs
  │                     ├─► library    MCP catalog, model data
  │                     └─► providers, MCP servers, APIs (through one SSRF-safe fetch)
  ├──► S3 (presigned uploads and downloads)
  └──► library (icons)

RepositoryCopy Anchor Link

TiersCopy Anchor Link

Every package has three entry points, and the linter enforces the boundaries:

• . — pure: types, zod schemas, constants. No React, no Node.
• ./client — browser code; may use React.
• ./server — Node code; may use pg, ioredis, the network.

A tier never imports another tier's client or server entry. The web app imports pure and client code (and server types); the API imports pure and server code. @kai/ai and @kai/mcp never import each other or @kai/auth; the API wires them together.

A message, start to finishCopy Anchor Link

1. POST /api/conversations/:id/messages. The API resolves the session and workspace, checks the agent's model and tool limits, resolves the toolset (asking each MCP server for its tools, from cache when fresh), stores the user message, and creates a run.
2. The run manager (one per process) claims the run. Its loop renders the provider request from stored rows, streams the reply, stores each part as it completes, and publishes events to a Redis stream.
3. Tool calls are checked against permissions and remembered approvals. Allowed calls run (up to 8 at once); others create an interrupt and the run waits, with no process attached, until someone answers.
4. The browser follows the run's events over SSE (GET /api/runs/:id/events), with Last-Event-ID so a reload or a dropped connection resumes where it left off. Any replica can serve the stream.
5. When the model stops asking for tools (or after 50 steps), the run ends. A background job writes the title.

Stored messagesCopy Anchor Link

Messages form a tree (each has a parent; edits and regenerations are siblings), and each message is a list of parts: text, reasoning, tool_call, tool_result, note, summary, attachment, provider_item.

Parts keep two representations: provider-neutral columns (text, tool name, input, status) and, for parts a model produced, the provider's raw block (provider_payload, with its provider and model). For the same provider Kai replays the raw block (thinking signatures, encrypted reasoning, web search results survive); for another provider it rebuilds from the neutral columns. So switching models mid-chat works, and no history is converted between provider formats.

The prompt cacheCopy Anchor Link

The rule that shapes the run engine: the request for step n+1 starts, byte for byte, with the request for step n, except at a new segment. The request is a pure function of stored rows: the system prompt and tools are snapshots per segment, changing facts are notes, and model output is replayed verbatim. Prompt caching explains how this works on each provider, including tool changes mid-conversation, and Context management how Kai keeps within a model's context window.

Data and tenancyCopy Anchor Link

Every row belongs to a workspace (an organization). The server connects as three roles:

SQL is hand-written and always parameterized. Migrations live in packages/core/migrations: auth.sql and schema.sql run once, functions.sql re-runs when it changes, rls.sql runs every time.

Secrets and the networkCopy Anchor Link

Secrets go through one vault (packages/core/secrets): AES-256-GCM, bound to their row and purpose. Every outbound request goes through one fetch (packages/core/net) that refuses private addresses unless allowed, pins the resolved address, and re-checks redirects. See Security.