Learn / Questions
Latency percentiles count the requests that finished. When the slow ones fail instead, the survivors look fast.
p99 latency is the response time that 99% of requests beat. The question is 99% of which requests. In most dashboards, and in this simulator, the percentile is computed over successful requests. A request that timed out or was rejected has no meaningful response time, so it is not in the sample.
Now put a database in trouble. The requests that needed it queue, time out and fail. The requests that did not need it, cache hits for example, finish in a few milliseconds as always. The failed requests leave the latency sample and the fast ones remain, so p99 falls at the exact moment users are getting errors. A graph of p99 alone would say the system got faster.
That is why success rate and latency have to be read together, and why the lab labels its latency numbers as "OK requests". A p99 of 30 ms with 60% success is an outage. A p99 of 400 ms with 99.9% success is a slow afternoon.
| What p99 covers | successful requests only |
|---|---|
| A failed request contributes | nothing to latency, one to failures |
| Read it with | success rate, and Apdex, which counts failures as dissatisfied |
Report latency next to success, or use a metric that charges failures, such as Apdex or "requests served within the target", which falls when requests fail fast.
Opening and running a problem is free. Changes by command cost 1 credit each; you have 25 free credits a day.