Streaming

Set stream: true and the reply arrives as server-sent events while the model writes it, instead of in one piece at the end. Every text endpoint streams, in the event format its SDK expects. Streamed requests are counted, billed and logged like any other.

Chat Completions

const stream = await client.chat.completions.create({
  model: "gpt-5.5",
  messages: [{ role: "user", content: "Write a haiku about latency." }],
  stream: true,
  stream_options: { include_usage: true },
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

stream = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "Write a haiku about latency."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

On the wire, each event is a data: line holding a chat.completion.chunk, and the stream ends with data: [DONE]:

curl -N https://api.cloakray.com/v1/chat/completions \
  -H "Authorization: Bearer $CLOAKRAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-5.5", "stream": true, "messages": [{"role": "user", "content": "Count to five."}]}'

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"One"}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

The last chunk before [DONE] carries usage — including the charged cost — and an empty choices array. Tool calls stream as delta.tool_calls fragments; join the arguments strings by index before parsing them.

Responses

The Responses API streams typed events. Text arrives in response.output_text.delta, and response.completed closes the stream with the full response and its usage, including cost:

const stream = await client.responses.create({
  model: "gpt-5.5",
  input: "Write a haiku about latency.",
  stream: true,
});

for await (const event of stream) {
  if (event.type === "response.output_text.delta") process.stdout.write(event.delta);
}

Messages

The Messages endpoint streams Anthropic's events: message_start, then content_block_start, content_block_delta and content_block_stop for each block, message_delta with the stop reason and output usage, and message_stop.

const stream = client.messages.stream({
  model: "claude-opus-4-8",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Write a haiku about latency." }],
});

stream.on("text", (text) => process.stdout.write(text));
await stream.finalMessage();

Errors during a stream

An error before the first token returns an ordinary HTTP error with a JSON body. Once the stream has started the status is already 200, so an error arrives as an event in the stream — an error object in a chunk, or an Anthropic error event. Handle both, and treat a stream that ends without its final event ([DONE], response.completed, message_stop) as incomplete.

What doesn't stream

Embeddings, images, speech, transcription and the other non-text endpoints return one response. Video is an asynchronous job you poll instead.