POST/v1/chat/completions

Chat Completions

Create a model response for a chat conversation — OpenAI dialect, with tools and vision.

Given a list of messages, the model produces a completion. Supports streaming (stream: true), tool calling, and vision (image content parts, on models with the vision capability).

Note

Only the parameters the gateway itself validates are listed below; every other OpenAI chat parameter is forwarded to the provider unchanged.

Request body

Field	Type	Required	Notes
`model`	string	yes	A curated alias (`gpt-4o`). Provider-native ids are rejected with 404 `model_not_found`.
`messages`	array	yes	OpenAI messages. Roles: `system`, `developer`, `user`, `assistant`, `tool`. Content may be a string or an array of parts.
`messages[].content[].image_url.url`	string	no	Must be a non-empty `http`/`https`/`data:` URL (format-checked only, never fetched). → 400 `invalid_image_url`.
`stream`	boolean	no	SSE streaming; see Streaming.
`stream_options.include_usage`	boolean	no	Always forced to `true` by the gateway — the final usage chunk is guaranteed.
`stop`	string \| string[] \| null	no	At most 4 sequences → 400 `invalid_stop_sequences`.
`max_tokens`	integer	no	Also sizes the prepaid credit reservation when the wallet check runs.
`max_completion_tokens`	integer	no	Forwarded to the provider unchanged.
`top_p`	number	no	Nucleus sampling. Must be in `(0, 1]` → a numeric value outside that range (notably `0`, an empty nucleus) is 400 `invalid_request` (`param: "top_p"`). Omit for the model default.
`temperature`, `n`	number / integer	no	Forwarded to the provider unchanged.
`tools`, `tool_choice`	array / object	no	Forwarded to the provider unchanged. `tool_call_id` on tool messages must match a preceding assistant `tool_calls[].id` → 400 `invalid_tool_call_id`.
`response_format`	object	no	Forwarded unchanged (e.g. JSON mode / JSON schema).
`user`	string	no	Forwarded to the provider unchanged.

Request validation (before the provider is called)

Trigger	Status	Code
`stop` with more than 4 sequences	400	`invalid_stop_sequences`
`top_p` outside `(0, 1]` (e.g. `0`)	400	`invalid_request` (`param: "top_p"`)
`tool_call_id` not matching a preceding assistant `tool_calls[].id`	400	`invalid_tool_call_id`
Malformed `image_url.url`	400	`invalid_image_url`
Image content on a non-vision model	400	`vision_not_supported` (`param: "model"`)
Model mode is not chat	400	`unsupported_endpoint` (`param: "model"`)

Code samples

curl https://api.<domain>/v1/chat/completions \
  -H "Authorization: Bearer $TOKENFACTORY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is the meaning of life?"}
    ]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://api.<domain>/v1",
    api_key=os.environ["TOKENFACTORY_API_KEY"],
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the meaning of life?"},
    ],
)
print(response.choices[0].message.content)
# provider_metadata is an unrecognized field for the OpenAI SDK, so it is kept
# as an extra: attribute access works, or use resp.provider_metadata["actual_model"].
print(response.provider_metadata.actual_model)

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'What is the meaning of life?' },
  ],
});
console.log(response.choices[0].message.content);

Set "stream": true to receive Server-Sent Events. Every data: record is a chat.completion.chunk; the stream ends with a chunk carrying the final usage (the gateway always includes it) and data: [DONE]:

curl https://api.<domain>/v1/chat/completions \
  -H "Authorization: Bearer $TOKENFACTORY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Count to three"}],
    "stream": true
  }'

data: {"id":"chatcmpl-9","object":"chat.completion.chunk","created":1724680009,"model":"gpt-4o","choices":[{"index":0,"delta":{"role":"assistant","content":"One"},"finish_reason":null}]}

data: {"id":"chatcmpl-9","object":"chat.completion.chunk","created":1724680009,"model":"gpt-4o","choices":[{"index":0,"delta":{"content":", two, three"},"finish_reason":null}]}

data: {"id":"chatcmpl-9","object":"chat.completion.chunk","created":1724680009,"model":"gpt-4o","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":8,"completion_tokens":6,"total_tokens":14,"prompt_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"completion_tokens_details":{"reasoning_tokens":0}}}

data: [DONE]

The same normalized usage shape described above (Normalized usage shape) applies to this terminal chunk — always present, 0-defaulted, never invented — regardless of how the upstream provider framed its own usage payload. Full stream contract (guarantees, side-channel headers, in-band errors): Streaming.

Tool calling

Tools are forwarded to the provider as sent. The assistant's finish_reason becomes tool_calls and the delta carries the tool invocation; send the tool's result back as a role: "tool" message with the matching tool_call_id:

curl https://api.<domain>/v1/chat/completions \
  -H "Authorization: Bearer $TOKENFACTORY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "What is the weather in Oslo?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'

Response (finish_reason: "tool_calls", invocation in message.tool_calls[]):

{
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "id": "call_1",
        "type": "function",
        "function": { "name": "get_weather", "arguments": "{\"city\":\"Oslo\"}" }
      }]
    },
    "finish_reason": "tool_calls"
  }],
  "usage": { "prompt_tokens": 42, "completion_tokens": 18, "total_tokens": 60 },
  "provider_metadata": { "provider": "openai", "actual_model": "gpt-4o-2024-08-06" }
}

Feed the result back as a tool message to continue the conversation — its tool_call_id must match (orphan ids are rejected: 400 invalid_tool_call_id, table above).

{
  "model": "gpt-4o",
  "messages": [
    { "role": "user", "content": "What is the weather in Oslo?" },
    { "role": "assistant", "tool_calls": [{ "id": "call_1", "type": "function", "function": { "name": "get_weather", "arguments": "{\"city\":\"Oslo\"}" } }] },
    { "role": "tool", "tool_call_id": "call_1", "content": "4°C, light snow" }
  ],
  "tools": []
}

Response

{
  "id": "chatcmpl-1",
  "object": "chat.completion",
  "created": 1724680001,
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "Hi", "refusal": null },
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 5,
    "completion_tokens": 1,
    "total_tokens": 6,
    "prompt_tokens_details": { "cached_tokens": 0, "cache_write_tokens": 0 },
    "completion_tokens_details": { "reasoning_tokens": 0 }
  },
  "system_fingerprint": "fp_1",
  "service_tier": "default",
  "provider_metadata": { "provider": "openai", "actual_model": "gpt-4o-2024-08-06" }
}

Guaranteed fields: id, object: "chat.completion", created, model (your alias), choices[].{index, message, finish_reason}, usage (when the provider reported it), and provider_metadata (always).

Normalized usage shape

Different upstream providers (and even different open-weight backends behind the same alias) report token usage differently — some send prompt_tokens_details/completion_tokens_details, some omit them entirely, some send an explicit null. The gateway normalizes all of this into one consistent shape, so your code never has to branch on which backend served a given request:

usage.prompt_tokens_details.cached_tokens
usage.prompt_tokens_details.cache_write_tokens
usage.completion_tokens_details.reasoning_tokens

are always present in the response body, defaulting to 0 when the provider didn't report that detail. When a provider did report a real value, it is passed through unchanged — the gateway never invents or estimates a nonzero number; 0 is only ever a safe "not reported" default, the same convention OpenRouter uses for its own normalized usage schema.

Provider-internal fields (nvext, native_finish_reason, timing keys) are stripped or relocated into provider_metadata.

Reasoning models return their reasoning text in choices[].message.reasoning_content (and choices[].delta.reasoning_content when streaming), next to content — the same field DeepSeek, vLLM and OpenRouter use. An upstream message.reasoning string is renamed to reasoning_content. provider_metadata.reasoning_content still carries a copy of choice 0's reasoning on non-streaming responses but is deprecated; read the message field.

Errors

Standard error surface: 400, 401, 402, 403, 404 (model_not_found), 410 (model_deprecated), 413, 429, 502, 503, 504 — see Errors for the envelope and full code table.