Chat completions

Endpoint and authentication

Send a JSON POST request with a bearer token. Use an API key issued for the intended workspace and environment.

POST https://api.octoryn.dev/v1/chat/completions
Authorization: Bearer $OCTORYN_API_KEY
Content-Type: application/json

Request body

model and messages are required. The Router accepts standard OpenAI-compatible optional fields and passes supported fields to the eligible route.

• model — a direct model identifier or policy/* alias
• messages — ordered role/content messages; multimodal content depends on route capability
• stream — false by default; true returns server-sent events
• max_tokens and temperature — optional generation controls
• tools, tool_choice and response_format — accepted when the selected route supports them
• Extra compatible fields are preserved where the route and provider support them

{
  "model": "policy/frontier",
  "messages": [
    {"role": "system", "content": "Answer concisely."},
    {"role": "user", "content": "Compare the two deployment options."}
  ],
  "temperature": 0.2,
  "stream": false
}

Non-streaming response

A successful non-streaming response uses the OpenAI chat completion envelope with id, object, created, model, choices and usage fields. Provider-specific additions may be present and clients should ignore fields they do not use.

{
  "id": "chatcmpl_...",
  "object": "chat.completion",
  "model": "policy/frontier",
  "choices": [{
    "index": 0,
    "message": {"role": "assistant", "content": "..."},
    "finish_reason": "stop"
  }],
  "usage": {"prompt_tokens": 24, "completion_tokens": 96, "total_tokens": 120}
}

Streaming

Set stream=true to receive text/event-stream. Each data line contains a chat.completion.chunk object. The stream ends with data: [DONE]. Consume incrementally and handle cancellation, timeout and partial output.

data: {"id":"chatcmpl_...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: [DONE]

Correlation and safe retries

Responses include available request correlation headers such as X-Request-Id. Record the identifier with status and latency, but never log API keys or sensitive prompt content. Retry only safe requests and keep the total attempt budget inside the user-facing deadline.