KaiTesting
Unit tests, the row-level security suite, and what each one guards.
| Command | What it runs |
|---|---|
pnpm test (make test) | Unit tests in every package (Vitest, through vp test). No services needed. |
make test-rls | The row-level security suite, against its own throwaway Postgres. |
make test-live | A few real provider calls. Needs ANTHROPIC_API_KEY and/or OPENAI_API_KEY; costs a few cents. |
pnpm typecheck | Every package and tier. |
pnpm lint | Lint, including the import boundaries between tiers and packages. |
Run the unit tests, the RLS suite, typecheck and lint before sending a change; run make test-live when you touch a provider adapter or the renderers.
Unit tests
| File | Guards |
|---|---|
packages/ai/tests/prefix.test.ts | The prompt cache: rendering is deterministic, each step's request starts with the previous one through tool loops and branches, segments and compaction start cleanly, notes and cache points land in the right place, and a chat moved between providers carries none of the other provider's payloads. |
packages/ai/tests/registry.test.ts | Model capabilities and defaults per provider. |
packages/core/tests/net.test.ts | The safe fetch: private, loopback, link-local and mapped addresses are refused; redirects are re-checked. |
packages/core/tests/secrets.test.ts | The vault: round trips; a ciphertext moved to another row, read for another purpose or tampered with fails; hints show only the last four characters. |
packages/core/tests/migrate-plan.test.ts | Which migration files run, and when. |
packages/mcp/tests/mcp.test.ts | The MCP host against in-process servers: transports, tool lists, errors. |
packages/mcp/tests/fields.test.ts | Header and environment fields: placeholders, formats, KAI_ENV_ only. |
packages/mcp/tests/openapi.test.ts | OpenAPI documents to tools, and requests from tool calls. |
packages/email/tests/email.test.ts | Email adapters and templates. |
Row-level security
tests/rls starts Postgres 17 in Docker (tests/rls/docker-compose.test.yml),
creates the roles, applies the real migrations, seeds two workspaces with
owners, admins and members, and then checks, as each role, what every table
lets them read and write.
make test-rls # bootstrap, run, teardown
STEP=bootstrap make test-rls # then STEP=run as often as you like, then STEP=teardown| Spec | |
|---|---|
owner-data.test.ts | Personal rows (chats, messages, personal credentials): only their owner sees them, in the active workspace; admins can't read members' rows; rows can't be moved to another user or workspace. |
workspace-config.test.ts | Workspace rows (keys, servers, tool groups, settings): members read them, only owners and admins write them, never across workspaces. |
agents.test.ts | The default agent, members' own agents, and who can change workspace agents. |
settings-and-roles.test.ts | Missing settings, admin checks matching the app's, kai_app having no BYPASSRLS or direct table privileges, and settings not leaking between pooled connections. |
server-only.test.ts | Tables only kai_service or kai_auth may touch. |
migration-catalog.test.ts | Every table has RLS enabled, belongs to exactly one class, and has that class's policies. |
A new table needs policies in rls.sql and a case in these specs.
Live provider tests
packages/ai/tests/live runs, for each provider with a key, a real tool loop
and a second turn: the model calls a tool, Kai replays its output exactly as it
stores it (thinking signatures, reasoning items, tool calls) with the result,
and then answers again. It checks the providers accept the replay and
that the later requests read from the prompt cache.
export ANTHROPIC_API_KEY=… OPENAI_API_KEY=…
make test-liveKAI_LIVE_ANTHROPIC_MODEL (default claude-haiku-5-5) and
KAI_LIVE_OPENAI_MODEL (default gpt-5.4-mini) pick the models.
KAI_LIVE_<ANTHROPIC|OPENAI>_BASE_URL and _API_KEY run the same checks
through a gateway:
KAI_LIVE_ANTHROPIC_BASE_URL=http://127.0.0.1:8080/anthropic \
KAI_LIVE_OPENAI_BASE_URL=http://127.0.0.1:8080/openai/v1 \
KAI_LIVE_ANTHROPIC_API_KEY=<virtual key> KAI_LIVE_OPENAI_API_KEY=<virtual key> \
make test-liveA second test checks tool search, and on Anthropic a tool change in the middle of a chat.
By hand
The fakes in scripts/ (see Local setup)
cover tool calls, approvals and scripted replies without spending tokens.
For real models, add a key in the app. The usage page's cache hit ratio is the
quickest check that a change didn't break caching.