Create a chat completion

curl --request POST \
  --url https://{host}/v2/workspaces/slug:llm-gateway/webhooks/v1/chat/completions \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "messages": [
    {
      "content": "<string>",
      "name": "<string>",
      "tool_call_id": "<string>",
      "tool_calls": [
        {
          "id": "<string>",
          "type": "function",
          "function": {
            "name": "<string>",
            "arguments": "<string>"
          }
        }
      ],
      "reasoning_details": [
        {
          "thinking": "<string>",
          "signature": "<string>",
          "data": "<string>"
        }
      ],
      "reasoning_model": "<string>"
    }
  ]
}
'

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    messages: [
      {
        content: '<string>',
        name: '<string>',
        tool_call_id: '<string>',
        tool_calls: [
          {
            id: '<string>',
            type: 'function',
            function: {name: '<string>', arguments: '<string>'}
          }
        ],
        reasoning_details: [{thinking: '<string>', signature: '<string>', data: '<string>'}],
        reasoning_model: '<string>'
      }
    ]
  })
};

fetch('https://{host}/v2/workspaces/slug:llm-gateway/webhooks/v1/chat/completions', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

import requests

url = "https://{host}/v2/workspaces/slug:llm-gateway/webhooks/v1/chat/completions"

payload = { "messages": [
        {
            "content": "<string>",
            "name": "<string>",
            "tool_call_id": "<string>",
            "tool_calls": [
                {
                    "id": "<string>",
                    "type": "function",
                    "function": {
                        "name": "<string>",
                        "arguments": "<string>"
                    }
                }
            ],
            "reasoning_details": [
                {
                    "thinking": "<string>",
                    "signature": "<string>",
                    "data": "<string>"
                }
            ],
            "reasoning_model": "<string>"
        }
    ] }
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

{
  "id": "<string>",
  "object": "chat.completion",
  "created": 123,
  "model": "<string>",
  "choices": [
    {
      "index": 123,
      "message": {
        "role": "system",
        "content": "<string>",
        "name": "<string>",
        "tool_call_id": "<string>",
        "tool_calls": [
          {
            "id": "<string>",
            "type": "function",
            "function": {
              "name": "<string>",
              "arguments": "<string>"
            }
          }
        ],
        "reasoning_details": [
          {
            "type": "thinking",
            "thinking": "<string>",
            "signature": "<string>",
            "data": "<string>"
          }
        ],
        "reasoning_model": "<string>"
      },
      "finish_reason": "<string>"
    }
  ],
  "usage": {
    "prompt_tokens": 123,
    "completion_tokens": 123,
    "completion_tokens_details": {
      "reasoning_tokens": 123
    },
    "cache_read_input_tokens": 123,
    "cache_write_input_tokens": 123,
    "total_tokens": 123,
    "cost": 123,
    "duration_ms": 123,
    "carbon": {}
  },
  "failover": {
    "from_model": "<string>",
    "to_model": "<string>"
  }
}

{
  "error": "<string>",
  "message": "<string>",
  "code": "<string>",
  "details": {},
  "status": 123,
  "retryAfter": 123,
  "provider": "<string>",
  "model": "<string>",
  "provider_error_type": "<string>"
}

{
  "error": "<string>",
  "message": "<string>",
  "code": "<string>",
  "details": {},
  "status": 123,
  "retryAfter": 123,
  "provider": "<string>",
  "model": "<string>",
  "provider_error_type": "<string>"
}

{
  "error": "<string>",
  "message": "<string>",
  "code": "<string>",
  "details": {},
  "status": 123,
  "retryAfter": 123,
  "provider": "<string>",
  "model": "<string>",
  "provider_error_type": "<string>"
}

{
  "error": "<string>",
  "message": "<string>",
  "code": "<string>",
  "details": {},
  "status": 123,
  "retryAfter": 123,
  "provider": "<string>",
  "model": "<string>",
  "provider_error_type": "<string>"
}

Completions

Create a chat completion

OpenAI-compatible chat completions. Routes the request through the gateway’s provider layer (OpenAI, Azure OpenAI, Anthropic, Vertex, Bedrock, OpenAI-compatible) based on the resolved model spec.

Streaming. When stream: true, the response is a text/event-stream of OpenAI-compatible delta chunks (ChatCompletionChunk) followed by a literal data: [DONE] payload, then one final chat.completion frame (same shape as ChatCompletionResponse) carrying the assembled message, finish_reason and usage. Provider-native stream shapes (Anthropic, Bedrock, Vertex) are normalised to OpenAI deltas before being forwarded. Provider errors mid-stream are emitted as a synthetic chunk carrying an error object and a content fallback message, followed by [DONE].

Failover. When the requested model was attempted and failed, and a fallback answered instead (governance failover mapping merged with fallback_models), the switch is reported as failover: { from_model, to_model } on the response body, or on the first forwarded chunk when streaming.

Thinking blocks. Anthropic thinking blocks are never streamed. They are returned on the assistant message as reasoning_details, tagged with reasoning_model (the model that answered). Send both back verbatim when replaying the turn: the gateway forwards the blocks only to that same model, drops them for any other, and never sends the reasoning_model tag to a provider.

When stream: false (default), the response is a single JSON ChatCompletionResponse. The non-streaming response is enriched with usage.cost, usage.duration_ms, and usage.carbon (Prisme.ai extensions over the standard OpenAI shape).

Prisme.ai extensions in the request body:

task_id - opaque correlation identifier for A2A flows.
analytics_context - caller-supplied context (orgSlug, agent_id, user_id, context_id, agent_allowed_models, call_type, message_turn) used to enrich analytics.llm.completion events.
fallback_models - ordered models tried when the requested model fails, merged with the governance failover mapping; each one is checked against the caller’s access like the primary.

Rate limiting. 100 requests per 60 seconds per consumer (auth.user_id or session.id).

Governance. Calls may be rejected with 403 MODEL_NOT_ALLOWED or 429 quota errors based on the caller’s organization governance (resolved via ai-governance-v2), or with 403 HEADER_POLICY_VIOLATION when a required upstream identity header would reach the model provider empty (see header_policy).

POST

chat

completions

Create a chat completion

curl --request POST \
  --url https://{host}/v2/workspaces/slug:llm-gateway/webhooks/v1/chat/completions \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "messages": [
    {
      "content": "<string>",
      "name": "<string>",
      "tool_call_id": "<string>",
      "tool_calls": [
        {
          "id": "<string>",
          "type": "function",
          "function": {
            "name": "<string>",
            "arguments": "<string>"
          }
        }
      ],
      "reasoning_details": [
        {
          "thinking": "<string>",
          "signature": "<string>",
          "data": "<string>"
        }
      ],
      "reasoning_model": "<string>"
    }
  ]
}
'

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    messages: [
      {
        content: '<string>',
        name: '<string>',
        tool_call_id: '<string>',
        tool_calls: [
          {
            id: '<string>',
            type: 'function',
            function: {name: '<string>', arguments: '<string>'}
          }
        ],
        reasoning_details: [{thinking: '<string>', signature: '<string>', data: '<string>'}],
        reasoning_model: '<string>'
      }
    ]
  })
};

fetch('https://{host}/v2/workspaces/slug:llm-gateway/webhooks/v1/chat/completions', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

import requests

url = "https://{host}/v2/workspaces/slug:llm-gateway/webhooks/v1/chat/completions"

payload = { "messages": [
        {
            "content": "<string>",
            "name": "<string>",
            "tool_call_id": "<string>",
            "tool_calls": [
                {
                    "id": "<string>",
                    "type": "function",
                    "function": {
                        "name": "<string>",
                        "arguments": "<string>"
                    }
                }
            ],
            "reasoning_details": [
                {
                    "thinking": "<string>",
                    "signature": "<string>",
                    "data": "<string>"
                }
            ],
            "reasoning_model": "<string>"
        }
    ] }
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

{
  "id": "<string>",
  "object": "chat.completion",
  "created": 123,
  "model": "<string>",
  "choices": [
    {
      "index": 123,
      "message": {
        "role": "system",
        "content": "<string>",
        "name": "<string>",
        "tool_call_id": "<string>",
        "tool_calls": [
          {
            "id": "<string>",
            "type": "function",
            "function": {
              "name": "<string>",
              "arguments": "<string>"
            }
          }
        ],
        "reasoning_details": [
          {
            "type": "thinking",
            "thinking": "<string>",
            "signature": "<string>",
            "data": "<string>"
          }
        ],
        "reasoning_model": "<string>"
      },
      "finish_reason": "<string>"
    }
  ],
  "usage": {
    "prompt_tokens": 123,
    "completion_tokens": 123,
    "completion_tokens_details": {
      "reasoning_tokens": 123
    },
    "cache_read_input_tokens": 123,
    "cache_write_input_tokens": 123,
    "total_tokens": 123,
    "cost": 123,
    "duration_ms": 123,
    "carbon": {}
  },
  "failover": {
    "from_model": "<string>",
    "to_model": "<string>"
  }
}

{
  "error": "<string>",
  "message": "<string>",
  "code": "<string>",
  "details": {},
  "status": 123,
  "retryAfter": 123,
  "provider": "<string>",
  "model": "<string>",
  "provider_error_type": "<string>"
}

{
  "error": "<string>",
  "message": "<string>",
  "code": "<string>",
  "details": {},
  "status": 123,
  "retryAfter": 123,
  "provider": "<string>",
  "model": "<string>",
  "provider_error_type": "<string>"
}

{
  "error": "<string>",
  "message": "<string>",
  "code": "<string>",
  "details": {},
  "status": 123,
  "retryAfter": 123,
  "provider": "<string>",
  "model": "<string>",
  "provider_error_type": "<string>"
}

{
  "error": "<string>",
  "message": "<string>",
  "code": "<string>",
  "details": {},
  "status": 123,
  "retryAfter": 123,
  "provider": "<string>",
  "model": "<string>",
  "provider_error_type": "<string>"
}

Authorizations

Authorization

string

header

required

User-bound credential carrying an identity: either a session JWT or a user access token (at:*) generated from the user settings UI. Send as Authorization: Bearer <token>. Org API keys (iak_*) are not accepted here - they carry no user identity. Use the x-prismeai-api-key header instead (see OrgApiKeyAuth).

Body

application/json

OpenAI-compatible chat completion request. Only the fields actually accepted by the gateway are documented here.

messages

object[]

required

Conversation history (system + user/assistant/tool turns).

Hide child attributes

messages.role

enum<string>

required

Author role.

Available options:

system,

user,

assistant,

tool

messages.content

Message content. Either a plain string, or an array of typed parts (e.g. text + image) for multimodal inputs.

messages.name

string

Optional author name (e.g. function name for tool messages).

messages.tool_call_id

string

For role: tool, the id of the tool call this message responds to.

messages.tool_calls

object[]

Tool calls emitted by an assistant message.

Hide child attributes

messages.tool_calls.id

string

required

Provider-issued tool call identifier.

messages.tool_calls.type

enum<string>

required

Tool kind. Currently always function.

Available options:

function

messages.tool_calls.function

object

required

Hide child attributes

messages.tool_calls.function.name

string

required

Function name.

messages.tool_calls.function.arguments

string

JSON-encoded arguments. Sent as a string (matches OpenAI's wire format).

messages.reasoning_details

object[]

Anthropic thinking blocks produced with this assistant message (thinking blocks carry thinking and signature, redacted_thinking blocks carry data). Returned by the gateway and meant to be sent back verbatim when the turn is replayed.

Hide child attributes

messages.reasoning_details.type

enum<string>

Available options:

thinking,

redacted_thinking

messages.reasoning_details.thinking

string

messages.reasoning_details.signature

string

messages.reasoning_details.data

string

messages.reasoning_model

string

Gateway model id that produced reasoning_details. On replay the blocks are forwarded only when this matches the model being called; the tag itself is never sent to a provider.

model

string

Model id from the catalogue (e.g. gpt-4o, eu.anthropic.claude-sonnet-4-20250514-v1:0, vertex-gemini-2.5-flash). Defaults to the gateway's defaults.completions model when omitted.

Maximum string length: 256

temperature

any

Sampling temperature (provider-dependent range, typically 0 to 2).

max_tokens

any

Max tokens to generate.

max_completion_tokens

any

Max tokens to generate, OpenAI's newer name for max_tokens. Either one is accepted; the gateway forwards a single max_tokens (the lower value when both are sent), capped to the model's limits.max_response_tokens. It is sent back as max_completion_tokens where the model requires it, through the model's rename_params or the provider's parameterAliases. A provider entry that excludes max_tokens without such a rename sends no output limit.

top_p

any

Nucleus sampling parameter.

frequency_penalty

any

OpenAI-style frequency penalty.

presence_penalty

any

OpenAI-style presence penalty.

stop

any

One or more stop sequences (string or array of strings).

stream

boolean

When true, the response is a text/event-stream of ChatCompletionChunk deltas terminating with data: [DONE].

tools

object[]

Tool/function definitions made available to the model. Forwarded to providers that support tool calling.

tool_choice

any

Tool selection hint: "auto", "none", "required", or { type: "function", function: { name } }.

response_format

object

OpenAI-style structured output hint (e.g. { "type": "json_object" }).

seed

any

Provider seed for reproducible sampling (where supported).

fallback_models

string[]

Prisme.ai extension. Ordered fallback models tried when the requested model fails (merged with the governance failover mapping). Each one is checked against the caller's access like the primary.

Maximum array length: 5

task_id

string

Prisme.ai extension. Opaque correlation id propagated to A2A (agent-to-agent) flows.

Maximum string length: 128

analytics_context

object

Prisme.ai extension. Caller-supplied analytics context merged into the analytics.llm.completion event.

Hide child attributes

analytics_context.orgSlug

string

analytics_context.agent_id

string

analytics_context.user_id

string

analytics_context.context_id

string

analytics_context.agent_allowed_models

string[]

analytics_context.call_type

string

analytics_context.message_turn

number

prompt_cache

boolean

Prisme.ai extension. When omitted, prompt caching is on for conversations, i.e. requests that carry prompt_cache_key or analytics_context.context_id, and off for one-shot calls, whose cache write would never be read back. true turns it on even without a key; false turns it off. Caching means cache markers (Claude, Nova) and a cache key (OpenAI, Azure GPT, Mistral — on Mistral the key is what enables caching). The automatic caching of OpenAI / Azure GPT and Gemini cannot be switched off and stays active.

prompt_cache_key

string

Prisme.ai extension (standard OpenAI field). Cache routing key sent to OpenAI, Azure GPT and Mistral. Defaults to analytics_context.context_id. Ignored when prompt_cache is false.

Maximum string length: 256

Response

Successful completion. Content type depends on request.stream:

application/json: non-streaming ChatCompletionResponse.
text/event-stream: SSE stream of ChatCompletionChunk payloads followed by data: [DONE] and a final chat.completion frame.

Non-streaming chat completion response. Mirrors OpenAI's shape with Prisme.ai extensions on usage (cost, duration_ms, carbon).

string

required

Generated id (chatcmpl-<correlationId>).

object

enum<string>

required

Available options:

chat.completion

created

integer

required

Unix timestamp (seconds).

model

string

required

Resolved model id used to serve the request.

choices

object[]

required

Hide child attributes

choices.index

integer

required

choices.message

object

required

A single message in a chat completion request or response.

Hide child attributes

choices.message.role

enum<string>

required

Author role.

Available options:

system,

user,

assistant,

tool

choices.message.content

Message content. Either a plain string, or an array of typed parts (e.g. text + image) for multimodal inputs.

choices.message.name

string

Optional author name (e.g. function name for tool messages).

choices.message.tool_call_id

string

For role: tool, the id of the tool call this message responds to.

choices.message.tool_calls

object[]

Tool calls emitted by an assistant message.

Hide child attributes

choices.message.tool_calls.id

string

required

Provider-issued tool call identifier.

choices.message.tool_calls.type

enum<string>

required

Tool kind. Currently always function.

Available options:

function

choices.message.tool_calls.function

object

required

Hide child attributes

choices.message.tool_calls.function.name

string

required

Function name.

choices.message.tool_calls.function.arguments

string

JSON-encoded arguments. Sent as a string (matches OpenAI's wire format).

choices.message.reasoning_details

object[]

Hide child attributes

choices.message.reasoning_details.type

enum<string>

Available options:

thinking,

redacted_thinking

choices.message.reasoning_details.thinking

string

choices.message.reasoning_details.signature

string

choices.message.reasoning_details.data

string

choices.message.reasoning_model

string

Gateway model id that produced reasoning_details. On replay the blocks are forwarded only when this matches the model being called; the tag itself is never sent to a provider.

choices.finish_reason

string

stop, length, tool_calls, content_filter, or another provider-specific value. Anthropic and Bedrock stop reasons are normalised: unfinished answers (max_tokens, model_context_window_exceeded, pause_turn) become length, refusals (refusal, guardrail_intervened, content_filtered) become content_filter, any other present value becomes stop.

usage

object

Token, cost, and carbon accounting.

Hide child attributes

usage.prompt_tokens

integer

Total input tokens, cached context included.

usage.completion_tokens

integer

Total output tokens, reasoning included. Providers that report thinking separately (Gemini's thoughtsTokenCount) have it folded in by their adapter, so the figure is comparable across providers. cost is computed from it.

usage.completion_tokens_details

object

Breakdown of completion_tokens. Present when the provider reports one (OpenAI reasoning models, Gemini/Vertex). Absent otherwise: Anthropic and Bedrock do reason but publish no separate count, so the thinking stays inside completion_tokens.

Hide child attributes

usage.completion_tokens_details.reasoning_tokens

integer

usage.cache_read_input_tokens

integer

Cached input tokens re-read on this call (0 when caching is off).

usage.cache_write_input_tokens

integer

Input tokens written to the provider cache (0 when caching is off or unbilled).

usage.total_tokens

integer

usage.cost

number<double>

Prisme.ai extension. Estimated USD cost computed from the model document's pricing. Prompt-cache tokens are priced at pricing.cache_read_per_1m_tokens / pricing.cache_write_per_1m_tokens (default: a provider ratio of the input price), the rest of the prompt at input_per_1m_tokens, the completion at output_per_1m_tokens.

usage.duration_ms

integer

Prisme.ai extension. Wall-clock duration of the call.

usage.carbon

object

Prisme.ai extension. Carbon-footprint estimate produced by _compute-carbon-footprint. details.region is the carbon region used and details.region_source the rule that picked it: model (the model's override), provider (the entry's carbon_region), bedrock-profile:<prefix>, bedrock:<aws region>, vertex:<location> or default (world).

failover

object

Reported when the requested model was attempted, failed, and a fallback answered instead. Absent when no switch happened.

Hide child attributes

failover.from_model

string

Model requested by the caller.

failover.to_model

string

Model that actually answered.

Introduction

Platform API

Agent Creator API

LLM Gateway API

Governance API

Knowledges API

Collections API

Create a chat completion

Authorizations

Body

Response