Provider Routing & Failover

How the gateway picks the provider behind a model alias, and what happens when one fails.

Every curated model alias is backed by one or more provider bindings — (provider, provider-side model id, region) rows owned by the control plane. This page explains how a request lands on a provider and what happens when that provider fails. It is written for API consumers; bindings themselves are managed by the TokenFactory operators, not through the public API.

The normal path

You send an alias. Requests match the gateway route for that alias (exact model-name match).
The gateway picks the binding configured as the model's route: the primary binding (lowest priority rank, then highest weight) serves the request.
The model name is virtualized. If the provider's internal model id differs from your alias, the gateway rewrites the model in the request body — your request shape never contains provider-internal names.
Usage, cost, and provider_metadata reflect what actually served. The response's provider_metadata.provider and actual_model tell you who served the request and their internal id (Models & Aliases).

Failover: priority tiers

A model with multiple enabled bindings is routed as ordered priority tiers:

Tries are ordered by the binding's priority (ascending — lower is more primary), then weight within a tier.
A lower-priority tier receives traffic only when every higher tier is unavailable — provider connect failures and gateway-class errors.
Within one tier with multiple bindings, traffic is distributed by weight (proportional).

Passive health detection backs this: after repeated consecutive gateway errors, a failing provider host is temporarily ejected from its tier so failover stops paying a failed first attempt on every request. Ejection is time-boxed; the provider is retried automatically. There are no active health probes hitting providers.

What triggers a failover

Failover is provider-side only. It is triggered by the provider being unreachable or returning gateway-class errors (5xx / connect failure) — not by your request's content, rate limits, or credit state (those are rejected up front with a 4xx and never silently rerouted).

What you see

Successful failover is transparent: the same request shape, the same alias echoed back, 200 — with provider_metadata telling you who actually served it.
No failover target available: you get a clean 502 provider_unavailable (provider credentials and internals are never exposed). Retry semantics are yours to own; the error carries the server-minted request_id (Errors).

What you cannot do (yet)

Pin a provider. Binding selection and priority are platform-operator decisions. There is no per-request provider hint.
Influence tiers. Priorities and weights are configured per model by the operator.

Pricing note

Each binding carries its own pricing, and usage is priced at the rate of the model alias — see Billing. Whether serving-provider changes under failover can alter per-token pricing is a catalog configuration question; your bill is always computed from the provider's own reported usage, never estimated.