Get started
Stream a response
Render partial output without hiding terminal state
Streaming improves perceived latency but adds cancellation, backpressure and incomplete-output states.
1. Request a stream
Set stream=true on an approved route and consume SSE incrementally.
2. Accumulate safely
Append text deltas, merge tool-call fragments by ID and retain finish_reason. Do not assume every event contains text.
3. Cancel abandoned work
Propagate browser or service cancellation to the backend and close the upstream response so cost and capacity are not wasted.
4. Recover visibly
Before output, classified transient failure may use Router fallback. After output begins, mark incomplete and let the user or application choose a visible retry.