Operate the gateway

Boot order

1. Load the config, from a file or from Postgres.
2. Validate every reference.
3. Initialize each plugin. Any initialization error refuses boot, regardless of required.
4. Bind the data plane and admin listeners.
5. Mark /readyz healthy and start accepting traffic.

That order is what makes a bad config safe to roll: the gateway refuses to serve rather than serving without a plugin you expected to be enforcing something.

Probes

Wire readinessProbe to /readyz and livenessProbe to /healthz. Plugin initialization is synchronous during boot; the CLI marks readiness after state construction rather than maintaining per-plugin health.

Config delivery

Config stores expose change streams, but the CLI does not currently subscribe AppState to them. A ConfigMap projection or Postgres write alone does not change active routes, providers, vaults, or plugins.

Validate before you ship

orca-gateway check /etc/orca-gateway/config.yaml

Run this in CI on every config change. It catches unresolvable destination references, duplicate plugin names, scope dimensions used but not declared, and malformed strategies - all of which would otherwise surface as a boot failure during a rollout.

Against a running gateway, orca-gateway admin config validate --file <path> runs the same checks through the admin API, which additionally exercises live checks such as whether a vault resolves.

Common incidents

All requests return 404 no_route. No compatible destination exists for the request. A model request that matches no configured route uses the lexicographically first compatible destination as route default, so inspect destination kinds as well as route matches.

The process is overloaded. server.max_concurrent_requests is not enforced by the current CLI server. Apply a concurrency ceiling at an ingress or proxy and scale replicas from observed latency and saturation.

A provider attempt times out. Set strategy.timeout_ms; server.request_timeout_ms is parsed but not applied as an end-to-end deadline. See Routes.

The process exits during boot. Plugin initialization errors fail state construction regardless of required. The boot logs name the plugin. A common cause is a config naming a plugin kind the binary does not implement, or a feature-gated backend in a binary built without that feature.

Rate limits admit more traffic than configured. token_bucket_local is enforced per replica. Use redis_window when all replicas must share one ceiling. See Rate limits.