Streaming

Streaming is standard OpenAI server-sent events: set "stream": true and read data: records terminated by data: [DONE]. /v1/messages streams in the native Anthropic event sequence.

Chat stream shape

Each data: record is an OpenAI chat.completion.chunk:

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","created":1724680001,"model":"gpt-4o","choices":[{"index":0,"delta":{"role":"assistant","content":"Hel"},"finish_reason":null}]}

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","created":1724680001,"model":"gpt-4o","choices":[{"index":0,"delta":{"content":"lo!"},"finish_reason":null}]}

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","created":1724680001,"model":"gpt-4o","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":5,"completion_tokens":3,"total_tokens":8,"prompt_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"completion_tokens_details":{"reasoning_tokens":0}}}

data: [DONE]

Three guarantees:

1. model in every chunk is your alias — chunks are otherwise passed through byte-identical, with model-name virtualization applied (upstream- private fields such as nvext are removed, as on non-streaming responses).
2. A final usage chunk is always present — on /v1/chat/completions the gateway forces stream_options.include_usage: true regardless of what you sent, so the terminating chunk carries usage. On the legacy /v1/completions it cannot add the flag, so a streaming request must send "stream_options": {"include_usage": true}; without it the request is rejected with 400 invalid_request (param: "stream_options.include_usage"), because the stream could not be metered.
3. That usage chunk has a normalized shape — usage.prompt_tokens_details.{cached_tokens,cache_write_tokens} and usage.completion_tokens_details.reasoning_tokens are always present on the chunk carrying usage, defaulting to 0 when the backend didn't report that detail, and passed through unchanged when it did. This is the same guarantee non-streaming responses get (see Chat Completions) — the gateway never invents a nonzero number, and it holds regardless of how the upstream backend framed its own SSE output, so you see one consistent usage shape no matter which model served the request.

Side-channel headers (set before the first byte)

On streams and audio bodies provider_metadata never appears as a trailing body field — it rides in these headers, which are set before the first byte. The same headers are also sent on plain JSON responses, so you can use one code path for both.

Detecting stream completion vs interruption

A stream ends in exactly one of two ways:

The in-band error record has the uniform error shape:

data: {"error":{"type":"server_error","message":"upstream stream interrupted","code":"stream_interrupted","request_id":"req_…"}}

Codes: stream_interrupted (the provider's stream was reset) or api_key_terminated (your key was revoked mid-stream).

The HTTP status is never rewritten after headers are sent — a failure after the first byte is always this in-band record, which is why the [DONE] check is the reliable completion signal.

Anthropic streaming (/v1/messages)

Streaming on the Messages dialect keeps Anthropic's native event sequence — message_start, content_block_start, content_block_delta, …, message_delta, message_stop — with the same side-channel headers. An interruption surfaces as the Anthropic-dialect error record and the stream ends without message_stop.

Related

• Chat Completions
• Errors