Latency Percentiles

Latency percentiles describe the distribution of request response times and expose slow-tail behavior that an average can hide. Common percentiles such as p50, p95, and p99 indicate the latency below which a given percentage of requests fall.

A percentile summarizes an ordered set of latency measurements. For example, p50 is the median: about half of requests are no slower than that value, while the other half are slower. p95 means about 95% of requests are at or below that latency, leaving roughly 5% slower.

Percentiles are useful because backend latency is often uneven. A service might have a low median while a small fraction of requests experience database contention, garbage collection, network delays, lock waits, or downstream slowness. The tail percentiles make those outliers visible instead of allowing them to disappear inside an average.

For a Java backend, latency should be measured around meaningful operations such as HTTP requests, database calls, and calls to downstream services. When investigating a high p99, break the request into stages and correlate latency with traffic, errors, resource usage, garbage collection, database behavior, and downstream dependencies.

Percentiles are not additive. You generally cannot calculate the p95 of an entire request by simply adding the p95 values of several components. The correct aggregate must be based on the distribution of complete request measurements, because slow events in different components may not occur on the same requests.

Percentiles also depend on the measurement window and calculation method. A reported p99 over one minute can differ substantially from a p99 over one day, and approximate monitoring systems may use different algorithms for estimating quantiles.

Key points

• p50 is the median latency.
• p95 means roughly 95% of observations are at or below that latency.
• p99 exposes behavior in the slow tail.
• Averages can hide a small population of very slow requests.
• Percentiles must be interpreted with their measurement window and aggregation method.
• Do not add component percentiles to estimate an end-to-end percentile.

Interview questions

Why can a p99 be misleading when the sample size is small?

A percentile is calculated from observations, so with very few requests the estimate has high statistical uncertainty. A single slow request can have a large effect on the upper tail of a small sample. The reported value also depends on the percentile calculation method. Therefore, a p99 should be interpreted together with request count and an appropriate time window.

A Java service has stable p50 latency but its p99 suddenly becomes much higher. How would you investigate it?

I would first confirm that the change is real and compare the same measurement window, traffic level, and endpoint population. Then I would correlate the tail with GC pauses, CPU saturation, thread or connection pool exhaustion, lock contention, database latency, downstream calls, and network behavior. Distributed traces can identify which stage is slow for affected requests. I would also check whether the increase coincides with a deployment, configuration change, traffic pattern, or dependency incident.

What do p50, p95, and p99 latency mean?

p50 is the median latency, so roughly half of the measured requests are at or below it. p95 is the latency threshold covering roughly 95% of observations, leaving about 5% above it. p99 similarly leaves about 1% of observations above the threshold. These higher percentiles are useful for seeing slow-tail behavior that a median can hide.

Why is percentile latency often more useful than average latency?

An average combines all observations into one value and can conceal a small group of very slow requests. A percentile shows where a chosen portion of the distribution lies, making slow-tail behavior easier to observe. For user-facing systems, those slow requests can still matter even if they represent a minority of traffic. In practice, engineers often examine several percentiles together with request volume and errors.

If service A has p95 latency of 50 ms and service B has p95 latency of 80 ms, is the combined p95 130 ms?

Not necessarily. Percentiles describe positions in distributions, and the slow observations in A and B do not necessarily occur on the same requests. Adding the two p95 values assumes a relationship between the component samples that may not exist. To determine end-to-end p95, measure the latency of complete requests and calculate the percentile from that distribution.