Latency in Performance Testing

Latency measures how much time an operation or request takes to complete. In Java backend systems, understanding latency helps engineers identify slow endpoints, database calls, network operations, and other performance bottlenecks.

Latency is the elapsed time between a defined starting event and the corresponding completion event. For an HTTP API, this commonly means the time from sending a request until receiving the response.

Latency is usually measured in units such as milliseconds. A single latency measurement can be useful for debugging, but performance testing normally examines many requests because response times vary with workload, system state, network conditions, and other factors.

Important statistics include average latency and percentiles such as p50, p95, and p99. The p95 value means 95% of measured requests completed at or below that latency, while the remaining 5% took longer. Percentiles are useful for seeing slow-tail behavior that an average can hide.

In a backend application, latency can come from several components, including application processing, database queries, external service calls, thread scheduling, garbage collection, and network communication. Reducing latency therefore usually requires measuring the relevant stages rather than optimizing code blindly.

Key points

• Latency is the elapsed time required for an operation to complete.
• Measure many requests because latency varies under real workloads.
• Use percentiles such as p50, p95, and p99 to understand the latency distribution.
• Backend latency can include application, database, network, and external-service time.
• Lower latency generally means requests complete faster, but optimization should be guided by measurements.

Interview questions

What is latency in a backend application?

Latency is the elapsed time needed for an operation to complete. For an HTTP endpoint, it can represent the time between a request being sent and its response being received. It is commonly measured in milliseconds. Lower latency means the operation completes in less elapsed time.

Why are p95 and p99 latency useful?

They describe the slower portion of the latency distribution. Averages can hide a small number of very slow requests, while p95 shows the latency below which 95% of observations fall and p99 does the same for 99%. These percentiles are therefore useful for understanding tail behavior.

How would you measure elapsed time for an operation in Java?

For elapsed-time measurement, I would use `System.nanoTime()` before and after the operation. I would subtract the starting value from the ending value and convert the result to the desired unit. This API is intended for measuring elapsed durations and should not be interpreted as a wall-clock timestamp.

An API's average latency is 50 ms, but p99 is 2 seconds. What could this indicate?

It indicates that most requests are fast but a small fraction experience very high latency. I would investigate the sources of tail latency rather than relying on the average. Possible causes include database contention, slow downstream calls, queueing, thread-pool exhaustion, garbage collection pauses, or unusual request inputs. I would correlate latency with traces, logs, resource metrics, and dependency timings.