Configuration reference
Every section and field of the configuration document for Orca AI Gateway, with defaults and the values the shipped binary supports.
The complete field reference. For how the pieces fit together, start at Configure the gateway.
Top-level parsing uses deny_unknown_fields, so an unrecognized key is a boot failure rather than a
silently ignored setting.
server
| Field | Type | Default | Notes |
|---|---|---|---|
listen | string | 0.0.0.0:8080 | Data plane bind address. |
admin_listen | string | 127.0.0.1:9099 | Admin API, metrics, and health probes. |
max_concurrent_requests | integer | 4096 | Parsed but not enforced by the current CLI server. Enforce concurrency at an ingress or proxy. |
request_timeout_ms | integer | 120000 | Parsed but not applied as an end-to-end deadline. Use per-route strategy.timeout_ms. |
tls.cert, tls.key | path | unset | Parsed and live-validated, but the current CLI still binds a plain TCP listener. Terminate TLS at an ingress or proxy in front of the gateway. |
identity
| Field | Type | Default | Notes |
|---|---|---|---|
scope_dims | list of string | [] | Tenancy dimensions. Every dimension used elsewhere must be declared here. |
validators[].name | string | - | Unique per gateway. |
validators[].kind | string | - | jwt or api_key. |
validators[].issuers, validators[].audiences | list | [] | Standard JWT validation for jwt. |
validators[].jwks_url | string | unset | Fetch signing keys from JWKS. |
validators[].static_public_key_pem | path | unset | Verify against a PEM key on disk. |
validators[].insecure_allow_any_issuer | boolean | false | Permit an empty JWT issuer list. Emits a startup warning. |
validators[].insecure_allow_any_audience | boolean | false | Permit an empty JWT audience list. Emits a startup warning. |
validators[].scope_from.jwt_claims | map | {} | claim to scope_dim. |
validators[].scope_from.header_map | map | {} | Accepted by the config model but not wired into validators. Raw X-Orca-Scope-* headers are handled separately by the request context. |
validators[].keys_file | path | - | Required for api_key. Contains SHA-256 key digests and principals. |
api_key accepts only Authorization: Bearer <key> and rejects a configured custom header.
See Identity for its key-file schema.
destinations
Keyed by destination name.
| Field | Type | Notes |
|---|---|---|
kind | string | mcp, openai, anthropic, azure_openai, openai_compatible, bedrock, vertex, native_api_key. |
base_url | string | Required for azure_openai and static mcp; aliased as url for mcp. Other model adapters have defaults. |
region | string | Optional AWS region for bedrock; defaults to us-east-1. |
credentials.vault | string | Required for every kind. |
credentials.credential_id | string | Optional logical credential selected through the named vault; defaults to vault name. |
model_map | map | Client model id to provider-native id. |
pricing.input_per_1m, pricing.output_per_1m | number | USD per million tokens. |
pricing.cache_read_per_1m, cache_write_per_1m, cache_write_1h_per_1m | number | Optional cache-token rates. |
pricing.reasoning_output_per_1m | number | Optional reasoning-output rate. |
pricing.per_request_usd | number | Optional fixed cost per request. |
destination_resolver | object | Dynamic destination lookup for the wildcard mcp destination. Only kind: http is supported. |
Adapter-specific fields are passed to the adapter. azure_openai also needs base_url; vertex
needs project and location; Azure deployment and api_version are optional. See
Model providers.
native_api_key instead requires provider, api, base_url, a nonempty allowed_paths list,
auth, and credentials.vault. Optional forward_headers, headers, and
allowed_query_params have explicit allowlists. It rejects model mapping and pricing fields; see
Native API-key destinations.
routes
| Field | Type | Default | Notes |
|---|---|---|---|
name | string | - | Appears in metrics and spans. |
match.path | string | any | Exact request path. |
match.model | glob | any | Exact, *, prefix, suffix, or contains match against model id. |
match.header | map | {} | All entries must match. |
match.scope | map | {} | Each pair must match the principal's scope. |
match.scope_kind | enum | equals | Accepted but not read by the route matcher; scope matching remains exact. |
idempotency_safe | boolean | true | When false, no retry and no fallback. |
strategy.mode | enum | - | fallback, loadbalance, conditional. |
strategy.targets[].destination | string | - | Must reference a declared destination. |
strategy.targets[].weight | integer | - | loadbalance requires all weights summing to 100, or no weights for equal split; 0 excludes a target. |
strategy.targets[].default_params, override_params, drop_params | - | - | Applied in that order. |
strategy.conditions[] | list | - | Used by conditional; each has if and target. |
strategy.retry.triggers | list | unset | http_5xx / 5xx, http_429 / 429, or timeout_ms / timeout. Unset means one attempt per target. |
strategy.retry.max_attempts, backoff_ms | integer | - | Per-target retry budget. |
strategy.timeout_ms | integer | unset | Per-attempt deadline. Set it explicitly because the server-wide timeout is not enforced. |
plugins
Every plugin instance shares these fields:
| Field | Type | Default | Notes |
|---|---|---|---|
name | string | - | Unique within its trait. |
kind | string | - | See the supported kinds below. |
required | boolean | true | Carried in config. Initialization errors currently fail state construction regardless of this value. |
failure_mode | enum | deny | deny, allow, or log_only. |
Supported kinds
| Slot | Kinds the binary wires up |
|---|---|
rate_limiter | token_bucket_local (alias token_bucket), redis_window |
cache | redis_exact; vector_semantic behind the cache-semantic feature |
cost_model | seed, static_table, http_refresh, registry, layered |
guardrails[] | pii_regex, llm_evaluator, ext_proc |
authorizers[] | yaml_acl, opa_http, ext_proc, allow_all |
usage_sinks[] | stdout, kafka, postgres, registry |
audit_sinks[] | noop, stdout, file, kafka, ext_proc |
trace_exporters[] | otlp, signed_tape |
signers[] | ed25519_local |
Kinds documented elsewhere but not implemented include token_bucket_redis,
openai_moderation, payload_log, and the parquet_s3 / otlp usage sinks. The cedar
authorizer is feature-gated and returns an error on every call even when enabled. postgres
requires the usage-postgres feature, which the CLI binary enables by default. An unsupported
kind fails at boot with an error listing what is available.
Cache backends can be constructed, but the request pipeline's cache lookup and store stages remain
placeholders. Configuring redis_exact or vector_semantic does not currently cache responses.
registry usage sends model-usage deltas to an Agent Engine registry. It intentionally skips MCP
events because the registry does not expose an MCP usage-ingestion contract. Any usage sink can add
a spool object with directory, max_events (default 10000), and retry_interval_ms (default
1000) for bounded local retry. Kafka audit and usage sinks also accept serialization.format: avro with a schema_registry block; JSON is the default.
rate_limits
| Field | Type | Notes |
|---|---|---|
match.scope | map | Both limiter kinds match scope only. Most matching scope dimensions win; ties use config order. |
qpm | integer | Requests per minute. |
tpm | integer | Tokens per minute, input plus output. |
qph | integer | Requests per hour. |
tph | integer | Tokens per hour. |
budget_usd_per_day | number | Deprecated daily spend cap. Use a user_daily_cost_budget policy guardrail for new configurations. |
budget_usd_per_month | number | Calendar-month spend cap. Requires destination pricing or plugins.cost_model. |
spend
| Field | Type | Default | Notes |
|---|---|---|---|
key_dims | list of string | [] | Verified scope dimensions in a limiter bucket. Empty uses every verified scope.<dim> attribute. |
trusted_header_dims | list of string | [] | Explicitly permits a header-derived scope value only when no verified value exists. |
request_tag_allowlist | list of string | [] | Client attribution-tag keys allowed into usage records. |
budget_timezone | IANA timezone | UTC | Calendar timezone for daily and monthly budget windows. |
headers.enabled | boolean | false | Emit rate-limit and budget headroom response headers. |
admission.default_max_output_tokens | integer | 4096 | Output-token estimate when request omits a maximum. |
admission.unpriced_model | enum | deny | deny, estimate, or allow_untracked. |
admission.estimate_usd_per_request | number | unset | Required when unpriced_model: estimate. |
See Rate limits for bucket matching and verified-identity behavior.
guardrail_source
Optional source for policy guardrail bundles. This is a top-level section, not a member of
plugins. It is required when guardrails_policy.mode is observe or enforce. Unlike the
top-level document, guardrail_source and guardrail_state accept unknown keys, so a misspelled key
such as cache_ttl_sec is ignored rather than rejected.
| Field | Type | Default | Notes |
|---|---|---|---|
name | string | - | Instance name. |
kind | enum | - | file or registry. |
path | path | - | Required for file. Reads a plain bundle only when unsigned input is explicitly allowed; otherwise reads an ES256 JWS. |
verification.allow_unsigned | boolean | false | Accept an unsigned file bundle. Use only in a trusted development environment. |
verification.public_key_file | path | unset | ES256 public keys in PEM form. |
verification.public_key_pem | string | unset | Inline ES256 public keys in PEM form. |
source_id | string | unset | Stable file-source identity for rollback protection. Required for signed sources. |
high_water_path | path | beside the bundle in unsigned development mode | Gateway-owned, persistent, writable high-water mark used to reject rollback. Required for signed sources. |
reload_interval_secs | integer | 120 | File polling fallback interval. |
base_url | URL | - | Required for registry. Registry internal-listener root. |
bearer_token_file | path | - | Required for registry; reread each time the gateway fetches a bundle. |
endpoint / path | string | /internal/v1/guardrails/effective | Registry endpoint override. |
timeout_ms | integer | 3000 | Registry request timeout. |
cache_ttl_secs | integer | 900 | How long a fetched bundle is reused before the registry is called again. A rule change can take this long to reach a session. |
on_backend_error | enum | last_known | deny, allow, or last_known, for a failed fetch when the session has no cached bundle. last_known behaves like deny, because a cached bundle is reused only within cache_ttl_secs. allow substitutes an allow-all bundle. |
Registry bundles are session-scoped and load lazily on the first authenticated request for that session. The gateway does not attempt an anonymous bundle fetch at startup.
The current CLI cannot initialize a signed file source on a fresh state volume because it does not expose the library's bootstrap operation. An uninitialized or lost high-water state fails closed.
guardrail_state
Optional state store for stateful policy guardrails. This is also a top-level section.
| Field | Type | Default | Notes |
|---|---|---|---|
name | string | - | Instance name. |
kind | enum | - | memory or redis. No direct registry state kind is wired. |
url | URL | - | Required for redis. |
key_prefix | string | orca: | Redis key prefix. |
session_ttl_secs, turn_ttl_secs | integer | 86400, 300 | Redis only. State lifetime for session and turn scopes. Values below 1 become 1. |
subject_day_ttl_secs, subject_minute_ttl_secs | integer | 172800, 300 | Redis only. State lifetime for subject windows. Values below 1 become 1. |
flush_interval_ms | integer | 2000 | Coalescing interval for registry writeback. |
When a Redis state store is paired with a registry guardrail source, the gateway builds a
write-back client from the source's base_url, bearer_token_file, and timeout_ms, and posts
state updates to /internal/v1/workspaces/{workspace}/sessions/{session}/guardrail-state. The
gateway fails to start unless base_url is a bare service root. Flush failures are logged, not
retried, and not returned to the client; when the flush queue is full, new batches are dropped.
guardrails_policy
Controls policy guardrail evaluation. Payload guardrails under plugins.guardrails[] use a
different contract.
| Field | Type | Default | Notes |
|---|---|---|---|
mode | enum | off | off, observe, or enforce. |
phases | list | all four Gateway phases | llm_request, llm_response, tool_call, tool_result. Harness-only request and response are rejected while policy is active. |
failure_mode.llm_request | enum | allow | deny, allow, or soft_allow_with_alert. |
on_missing | enum | allow | Behavior when no bundle is available. |
on_expired | enum | soft_allow_with_alert | Behavior once a bundle is older than its expires_at plus max_staleness_secs. At tool_call, anything other than allow denies. |
on_unsupported_guardrail | enum | deny | deny or skip_with_alert. |
refresh_interval_secs | integer | 30 | Bundle refresh cadence. Must be greater than zero. |
max_staleness_secs | integer | 900 | Grace period after a bundle's expires_at before on_expired applies. Must be greater than zero. |
vaults
| Field | Type | Default | Notes |
|---|---|---|---|
name | string | - | Referenced by credentials.vault. |
resolver | enum | - | env, static_file, http. |
env_var | string | unset | Fixed environment variable for env. When absent, the resolver derives a namespaced name. |
scheme | string | derived | Optional canonical credential scheme for env. |
prefix | string | ORCA_VAULT__ | For derived env names. |
file | path | - | For static_file. YAML or JSON nested credential map loaded at startup. |
url_template | string | - | For http. Interpolates {credential_id} and {scope.<dim>}. |
bearer_token_file | path | unset | Optional bearer token file for http. |
timeout_ms | integer | 5000 | For http. |
cache | object | enabled | enabled: true, ttl_secs: 300, max_entries: 1024. |
Cloud secret-manager resolvers are not implemented. Use an environment variable, static file, or
an http resolver in front of a trusted credential service. See Vaults
for the static-file schema and cache behavior.
legacy_mcp_gateway
Accepts the YAML of the earlier mcp-gateway service verbatim and expands it before validation.
Mutually exclusive with every other top-level key. Deprecated. See
Migrate.