Usage and audit records

Two record streams answer different questions. Usage records say what a request cost. Audit events say what each policy decided and why. Both are emitted through configured sinks; usage delivery is queued, while audit delivery can run on the caller's path.

Usage records

Usage records use schema "2". Each carries an event_id for downstream deduplication, request and trace ids, timestamp, status, principal and verified scope, destination, provider, route, model, the full token breakdown, latency, optional cost and pricing provenance, plus MCP-specific fields when applicable.

cost_usd is computed from the destination's pricing override or from plugins.cost_model. A request whose model cannot be priced follows spend.admission.unpriced_model; it is never silently recorded as zero cost. See Destinations.

plugins:
  usage_sinks:
    - name: local
      kind: stdout
      required: false
      failure_mode: log_only

The shipped binary accepts four usage sink kinds:

Parquet and OTLP are not available as usage sink kinds. OTLP trace export is a separate, supported plugins.trace_exporters[] capability.

plugins:
  usage_sinks:
    - name: usage-kafka
      kind: kafka
      brokers: "kafka-0:9092,kafka-1:9092"
      topic: orca-usage
      buffer: 10000
      required: false
      failure_mode: log_only

To retain failed deliveries locally and replay them after restart, add a bounded spool to any usage sink:

plugins:
  usage_sinks:
    - name: usage-kafka
      kind: kafka
      brokers: "kafka-0:9092,kafka-1:9092"
      topic: orca-usage
      spool:
        directory: /var/lib/orca-gateway/usage-spool
        max_events: 10000
        retry_interval_ms: 1000

Give each sink and replica its own spool directory on persistent storage. A spool preserves event_id for downstream deduplication but remains bounded: new failed events are dropped and counted after max_events is reached.

Kafka usage and audit sinks accept serialization.format: avro with a schema_registry block. Avro writes to <base-topic>-avro; its default subject is <base-topic>-avro-value. When a Kafka usage sink combines Avro with a spool, set ack: all and start with a separate empty spool directory so the persisted destination and encoding binding cannot drift.

Query Postgres usage

When a Postgres usage sink is configured, query its deduplicated ledger through the admin plane:

curl -fsS \
  'http://localhost:9099/admin/v1/spend/usage?dim.workspace_id=ws_42&group_by=provider,model'

The endpoint also accepts RFC 3339 from and to timestamps. Fixed grouping dimensions are provider, model, destination, route, and traffic_kind; custom verified scope dimensions use dim.<name>. Without a Postgres usage sink, the endpoint returns 503 usage_query_unavailable.

Audit events

Audit events record decisions rather than totals: which guardrail fired and what it decided, what the authorizer allowed or denied, and which route and destination were picked. Guardrail audit events include decision and patch metadata, not request or response bodies.

plugins:
  audit_sinks:
    - name: audit
      kind: kafka
      brokers: "kafka-0:9092,kafka-1:9092"
      topic_template: "orca.{scope.workspace_id}.audit"
      required: false
      failure_mode: log_only

Available kinds

Built-in audit sinks serialize schema "1": at, request_id, trace_id, traffic_kind, action, resource, decision, principal id and kind, principal attributes, scope, route name, and event attributes. Kafka keys each event by request_id; its topic_template can interpolate {scope.<dim>}. File sinks require path; Kafka sinks require brokers and topic_template.

Audit records still contain principal, scope, resource, and policy metadata. Give them a retention policy and access controls appropriate for tenant-attributed operational data.

Sink failures

Usage delivery stays off the hot path through bounded queues and the optional spool. Kafka audit waits up to its configured producer timeout. Do not use a sink's failure_mode as an audit durability control, and do not rely on a sink for a regulatory "record or reject" requirement.

Related