Configure the gateway

Orca AI Gateway is configured through a single YAML or JSON document. Configuration is parsed and validated as one unit. For model traffic, a request that matches no configured route uses the lexicographically first compatible destination as route default; 404 no_route means no compatible destination exists.

The document

server:        { ... }   # listeners, TLS, request budgets
identity:      { ... }   # tenancy dimensions and auth validators
destinations:  { ... }   # named upstream targets
routes:        [ ... ]   # match to strategy bindings
plugins:       { ... }   # named plugin instances
rate_limits:   [ ... ]   # per-scope quotas
spend:         { ... }   # rate-limit bucket identity and unpriced-model policy
guardrail_source: { ... } # file or registry policy bundle source
guardrail_state:  { ... } # memory or Redis policy state
guardrails_policy: { ... } # policy phases and enforcement mode
vaults:        [ ... ]   # secret resolvers

Each section has its own page:

Server settings

server:
  listen: 0.0.0.0:8080
  admin_listen: 127.0.0.1:9099
  max_concurrent_requests: 4096
  request_timeout_ms: 120000
  tls:
    cert: /etc/orca-gateway/tls/tls.crt
    key: /etc/orca-gateway/tls/tls.key

Terminate TLS and enforce process-wide concurrency at an ingress or proxy. Set routes[].strategy.timeout_ms for an enforced per-attempt provider deadline.

Validation

Run orca-gateway check <path> before shipping any change. Validation is structural and referential, not just schema-level. It confirms that:

• every routes[].strategy.targets[].destination resolves to a declared destination
• plugin instance names are unique within their trait
• every scope key used in a match.scope or a rate limit is declared in identity.scope_dims
• each strategy mode has the shape it requires - conditional needs conditions[], loadbalance needs weights
• the legacy and modern config shapes are not mixed

Errors name the offending YAML path, such as routes[2].strategy.targets[0].destination. Top-level parsing uses deny_unknown_fields, so a typo in a key is a boot failure rather than a silently ignored setting.

Applying changes

The config-store implementations expose file watching and Postgres revisions, but the current CLI loads one snapshot while constructing AppState and does not subscribe the running data plane to store changes. Restart or roll the gateway after a file edit or admin write. See Admin API and control plane for the persistence behavior.

Plugin instances

Every cross-cutting behavior is a named plugin instance, and they all share four fields:

failure_mode is the setting to think hardest about. deny fails the request closed, which is usually right for authorization and guardrails. log_only is appropriate for sinks, where losing a telemetry record should not fail a user's request.