Reverse proxy and TLS

Put Kai behind a proxy that terminates TLS, and set KAI_APP_URL to the URL people open.

KAI_APP_URL=https://kai.example.com
S3_PUBLIC_ENDPOINT=https://files.example.com
KAI_LIBRARY_PUBLIC_URL=https://library.example.com

Three things are reached by browsers, so each needs a public address:

StreamingCopy Anchor Link

Replies stream over server-sent events. Turn off response buffering for /api/runs/ and /api/events, and allow long-lived connections.

server {
  server_name kai.example.com;

  location / {
    proxy_pass http://127.0.0.1:3000;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    client_max_body_size 2m;
  }

  location ~ ^/api/(runs/[^/]+/events|events)$ {
    proxy_pass http://127.0.0.1:3000;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_buffering off;
    proxy_read_timeout 300s;
  }
}

kai.example.com {
  reverse_proxy 127.0.0.1:3000 {
    flush_interval -1
  }
}

If a stream drops, the browser reconnects and picks up where it left off, so a proxy that cuts idle connections is survivable, just slower.

Several replicasCopy Anchor Link

Kai can run as several kai containers behind one proxy. Runs are coordinated through Postgres and Redis: a reply started on one replica streams through any other. No sticky sessions are needed.

Kai needs a long-lived process: it doesn't run on serverless platforms.