Documentation
Streaming
Server-sent events, usage totals and what happens when a stream is interrupted.
Base URL for this deployment: https://www.aiteraiworld.com/api/v1
Server-sent events
Set stream to true. Each event is a chat.completion.chunk object, and the stream ends with [DONE]. Ask for token totals in the final chunk with stream_options.
{ "model": "author/model", "stream": true, "stream_options": { "include_usage": true }, "messages": [{ "role": "user", "content": "Write a haiku" }] }data: {"id":"gen-6f1c…","object":"chat.completion.chunk","choices":[{"delta":{"content":"Quiet"},"index":0}]}
data: {"id":"gen-6f1c…","object":"chat.completion.chunk","choices":[{"delta":{},"finish_reason":"stop","index":0}],"usage":{"prompt_tokens":12,"completion_tokens":17,"total_tokens":29}}
data: [DONE]Interruptions and billing
Failover happens before the first byte: if a provider fails to start the stream, the next eligible provider is tried and the attempt is free. If an upstream drops the connection after output has been delivered, the stream emits an error event containing finish_reason: "error" and the generation id.
Delivered output is billed at the catalog rate. The X-AiterAI-Cost header is zero while a stream is open, so read the final cost from GET /generation?id= once the record settles.
With the SDKs
stream = client.chat.completions.create(
model="author/model",
messages=[{"role": "user", "content": "Write a haiku"}],
stream=True,
stream_options={"include_usage": True},
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")