Tail Latency

ELI5 - The Vibe Check

Your average response time might look great, but tail latency is about that unlucky slice of requests, the slowest 1 percent or so, that take way longer than everyone else. Here's the kicker: if a single user action fans out to 50 backend calls, even a rare 1-in-100 slow call means most of those user actions will hit at least one slow call. Averages lie, the tail is where users actually feel pain.

Real Talk

Tail latency refers to the response times at the high percentiles of a distribution, typically p95, p99, or p999, which matter more than the mean for user experience because a single user request often fans out into many downstream calls, and the overall response time is bounded by the slowest of them. Systems with many parallel dependencies suffer 'tail latency amplification,' where even a small per-call tail probability compounds into a large fraction of end-to-end requests being slow, which is why SLOs are defined on percentiles, not averages.

When You'll Hear This

"P50 looks fine, but p99 tail latency is 4 seconds, that's what's actually driving the complaints." / "With 40 parallel fan-out calls, tail latency amplification means almost every request hits at least one slow one."