Streaming

ELI5 — The Vibe Check

Streaming is when the AI sends you its response word by word as it generates, instead of making you wait for the whole thing at once. You know that typing effect you see in ChatGPT and Claude? That's streaming. Without it, you'd stare at a loading spinner for 30 seconds, then get the whole essay dumped on you at once.

Real Talk

Streaming in LLM APIs means the model's output is returned as a sequence of chunks (tokens or token groups) via server-sent events (SSE) or websockets rather than a single response after full generation. This enables progressive UI rendering, faster time-to-first-token perception, and the ability to stop generation early.

Show Me The Code

with client.messages.stream(
    model="claude-opus-4-6",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Explain recursion"}]
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

When You'll Hear This

"Enable streaming so users see the response immediately." / "Streaming dropped latency perception by 80%."