Time to First Token
TTFT
ELI5 - The Vibe Check
Time to First Token is how long a user stares at a blank screen before the AI starts typing back. It doesn't matter how fast the model writes once it gets going if the user already gave up waiting for the first word. This is why chat products obsess over shaving milliseconds off the very start of a response, even more than the total response time.
Real Talk
Time to First Token measures the latency between a request being sent and the first output token being returned, dominated by the prefill phase where the entire prompt is processed before generation begins. It is tracked separately from total generation time and inter-token latency because prompt length, queuing, and cold starts affect it disproportionately, and it drives perceived responsiveness in streaming interfaces.
When You'll Hear This
"TTFT spikes whenever the context is over 50K tokens." / "We cache the system prompt to keep TTFT under 300ms."
Related Terms
Context Window
A context window is how much text an AI can 'see' at once — its working memory.
Inference
Inference is when the AI actually runs and generates output — as opposed to training, which is when it's learning.
Latency Budget
A latency budget is deciding upfront how many milliseconds each part of your AI pipeline is allowed to spend before the whole thing feels slow to a user.
Prompt Caching
Prompt caching is a speed and cost optimization where the AI remembers the beginning of your prompt so it doesn't have to re-process it every time.
Streaming
Streaming is when the AI sends you its response word by word as it generates, instead of making you wait for the whole thing at once.