Streaming

Streaming allows you to receive model responses incrementally as they are generated, rather than waiting for the entire response to complete. This is especially useful for chat interfaces and long-form content generation, where you want to display output to users in real time.

OpenModel supports streaming across all three API formats using Server-Sent Events (SSE).

Enabling Streaming

OpenAI Format

Set "stream": true in the request body. The response will be an SSE stream of data: lines, each containing a JSON chunk.

curl -N https://api.openmodel.ai/v1/responses \
  -H "Authorization: Bearer $OPENMODEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4",
    "input": "Write a haiku about programming.",
    "stream": true
  }'

Anthropic Format

Set "stream": true in the request body. The Anthropic streaming format uses named event types to structure the response.

curl -N https://api.openmodel.ai/v1/messages \
  -H "x-api-key: $OPENMODEL_API_KEY" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 1024,
    "stream": true,
    "messages": [
      {"role": "user", "content": "Write a haiku about programming."}
    ]
  }'

Other Messages protocol providers: DeepSeek, DashScope, and Xiaomi all support streaming with the same behavior and event format as Anthropic. Use "stream": true and the Anthropic SDK's streaming methods — the code is identical, just change the model name.

Gemini Format

For Gemini, use the streamGenerateContent endpoint instead of generateContent. Append ?alt=sse when you want Server-Sent Events that can be processed incrementally.

OpenModel preserves the response format requested by the client: alt=sse returns an SSE stream, while omitting alt (or using alt=json) returns the standard complete JSON array. The official Google GenAI SDK selects the appropriate format automatically. For a real-time UI or a command-line stream, use the SDK streaming method or request alt=sse explicitly.

curl -N "https://api.openmodel.ai/v1beta/models/gemini-3.5-flash:streamGenerateContent?alt=sse&key=$OPENMODEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "contents": [
      {"parts": [{"text": "Write a haiku about programming."}]}
    ]
  }'

Event Format Differences

Each API format structures its streaming events differently.

OpenAI Events

OpenAI streaming sends data: lines with JSON objects. The stream ends with data: [DONE].

data: {"type":"response.output_item.added",...}
data: {"type":"response.content_part.added",...}
data: {"type":"response.output_text.delta","delta":"Hello"}
data: {"type":"response.output_text.delta","delta":" world"}
data: {"type":"response.output_text.done",...}
data: {"type":"response.completed",...}

Anthropic Events

Anthropic uses named SSE event types with event: and data: fields:

event: message_start
data: {"type":"message_start","message":{...}}

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Hello"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" world"}}

event: content_block_stop
data: {"type":"content_block_stop","index":0}

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":12}}

event: message_stop
data: {"type":"message_stop"}

Gemini Events

Gemini SSE streams send data: lines where each chunk contains a candidates array:

data: {"candidates":[{"content":{"parts":[{"text":"Hello"}],"role":"model"}}]}

data: {"candidates":[{"content":{"parts":[{"text":" world"}],"role":"model"}}]}

data: {"candidates":[{"content":{"parts":[{"text":"!"}],"role":"model"},"finishReason":"STOP"}],"usageMetadata":{...}}

Token usage is normally reported in usageMetadata on the final chunk. Always consume the stream through its terminating event instead of stopping after the first text chunk; OpenModel uses the completed usage data for accurate billing and displays the result in your consumption logs.

Best Practices

Handle Partial Chunks

SSE data may arrive in partial TCP segments. Make sure your parsing logic buffers incoming data and splits on complete \n\n-delimited events rather than assuming each network read contains a full event.

Set Appropriate Timeouts

Streaming requests take longer than non-streaming requests. Configure your HTTP client with a longer read timeout (e.g., 5+ minutes) to avoid cutting off long-running generations:

import httpx
from openai import OpenAI

client = OpenAI(
    base_url="https://api.openmodel.ai/v1",
    api_key="your-api-key",
    timeout=httpx.Timeout(300.0, connect=10.0),
)

Handle Connection Drops

Network interruptions can terminate a stream mid-response. Implement reconnection logic in your application:

Catch connection errors and timeouts
For non-idempotent requests, consider saving the partial response and notifying the user
Use the SDK's built-in retry mechanisms when available

Buffer and Flush for UI

When rendering streamed text in a user interface, buffer incoming deltas and flush them to the display at regular intervals (e.g., using requestAnimationFrame) rather than re-rendering on every single delta event. This prevents excessive DOM updates and keeps the UI smooth.

Streaming

On this page