Gateways

A gateway sits between Kai and the providers: one place for keys, budgets, rate limits, logging and fallbacks. Kai works with any gateway that speaks the Anthropic Messages API or the OpenAI Responses API. You add it as a key in Settings → BYOK with a base URL (see BYOK).

How it fitsCopy Anchor Link

• One key per API format. Add the gateway twice: as an Anthropic key for Claude models, and as an OpenAI key for GPT models. Each speaks its provider's native API, so nothing Kai relies on is translated away (thinking, encrypted reasoning, cache breakpoints, tool search).
• The API key is whatever the gateway expects: a virtual key, a team key, or its master key. Kai sends it as x-api-key on Anthropic keys and as Authorization: Bearer on OpenAI keys. Anything else goes in Extra headers.
• A gateway on your own network (localhost, 10.x, a Compose service name) is a private address. The instance needs KAI_ALLOW_PRIVATE_NETWORK=true, and the key needs Allow private network.
• Models come from the gateway's model list. Gateways usually list everything they can route, across providers, so switch off the ones that don't belong to the key's format.
• Capabilities (context window, reasoning efforts, tool search) come from the model id. Kai recognizes provider ids, with or without a anthropic/ or openai/ prefix. Other names (aliases, deployment names) get the provider's defaults; the model's info card says so.

Prompt caching keeps working through a gateway that passes requests on unchanged. The cache hit ratio on Usage is the quickest check.

LiteLLMCopy Anchor Link

The LiteLLM proxy serves both formats at its root.

Kai sees LiteLLM's model_names, not the models behind them. Name them after the real model ids and Kai knows their capabilities:

model_list:
  - model_name: claude-sonnet-5-5 # Kai finds this one
    litellm_params:
      model: anthropic/claude-sonnet-5-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: gpt-5.4
    litellm_params:
      model: openai/gpt-5.4
      api_key: os.environ/OPENAI_API_KEY
  - model_name: work-fast # an alias: provider defaults
    litellm_params:
      model: anthropic/claude-haiku-5-5
      api_key: os.environ/ANTHROPIC_API_KEY

Both keys list all of them. Switch the GPT models off on the Anthropic key and the Claude models off on the OpenAI key, so each model is reached through its own API.

BifrostCopy Anchor Link

Bifrost serves each provider's native API under its own path.

On the OpenAI key, keep the openai/… models on and switch the rest off. Claude models belong on the Anthropic key, where Kai speaks Claude's own API.

Use a recent Bifrost release. Older ones rewrite the adaptive thinking that current Claude models need, and every reply fails with "thinking.type.enabled" is not supported for this model.

If Bifrost is set to drop unknown headers, allow anthropic-beta (Settings → Client settings). Otherwise Kai can't change tools in place in the middle of a chat; it notices the rejection, turns that feature off for the key, and re-reads the conversation instead when tools change.

Azure OpenAICopy Anchor Link

Other gatewaysCopy Anchor Link

Any gateway works if it supports, with streaming:

• Anthropic keys: POST <base URL>/v1/messages, passing content blocks through unchanged (thinking blocks and their signatures), with cache_control and the anthropic-beta header.
• OpenAI keys: POST <base URL>/responses, with store: false, include: ["reasoning.encrypted_content"] and prompt_cache_key.
• GET <base URL>/v1/models (Anthropic) or <base URL>/models (OpenAI) to list models. Without it, add model ids by hand.

Test connection in the key form lists the models with what you typed, so it's the first check.

When a gateway rejects a featureCopy Anchor Link

Kai uses a few optional features on every key: streaming tool input, tool search, web search, changing tools in place (Anthropic), and encrypted reasoning, reasoning summaries, the cache key and rich tool results (OpenAI). When a gateway rejects one with an error naming it, Kai turns it off for that key, retries once, and lists it under Advanced in the key's form with a reset button, for when the gateway is upgraded.

TroubleshootingCopy Anchor Link