System Design Bottleneck

A system design bottleneck is a component whose limited capacity constrains the throughput or latency of the overall system. Backend engineers identify bottlenecks by measuring resource usage and then improve, partition, cache, queue, or scale the constrained component.

A bottleneck occurs when one part of a request path cannot keep up with the workload placed on it. Common examples include a database, CPU, network link, connection pool, disk, external API, or a synchronized section of application code.

The important distinction is between a component that is busy and one that limits the system. A database running at 90% CPU may be a bottleneck if requests queue behind it, while another component at 95% utilization may not matter if it has enough spare capacity and does not constrain throughput.

In an interview, start by identifying the request path and its dependencies. Look for high latency, queue growth, saturation, exhausted connection pools, lock contention, or a resource approaching its capacity limit. Metrics such as throughput, latency percentiles, CPU, memory, I/O, queue depth, and database wait time help confirm the suspected bottleneck.

Typical remedies depend on the cause. Caching can reduce repeated reads, indexes can reduce database work, batching can reduce per-request overhead, asynchronous queues can absorb bursts, and horizontal scaling can add capacity. Partitioning or sharding may be needed when one instance or dataset has become the limiting resource.

A useful design principle is to avoid optimizing components that are not limiting the system. Fixing a non-bottleneck may improve a local metric without increasing end-to-end capacity.

Key points

• A bottleneck limits overall throughput or contributes materially to end-to-end latency.
• Common bottlenecks include databases, CPU, network, disk, connection pools, locks, and external services.
• Use measurements such as latency, throughput, utilization, queue depth, and wait time to validate a bottleneck.
• Choose the remedy based on the constraint: cache, index, batch, queue, partition, or scale.
• Optimizing a non-bottleneck may provide little system-level improvement.

Interview questions

What is a bottleneck in a backend system?

A bottleneck is a component whose capacity limits the system's overall performance. For example, if several application servers share a database that can handle fewer operations than the servers generate, the database can become the bottleneck. Requests may then experience increased latency or queue behind that resource. The correct fix depends on what is limiting capacity.

How would you find a bottleneck in a Java backend?

I would trace the request path and measure each important dependency rather than guessing from one metric. I would inspect throughput, latency percentiles, CPU, memory, database waits, connection-pool usage, queue depth, and network or disk I/O. I would compare those measurements with known capacity limits and reproduce the workload when possible. The goal is to establish that the suspected component actually constrains the system.

If the database is the bottleneck, what solutions would you consider?

First I would determine whether the workload is read-heavy, write-heavy, CPU-bound, I/O-bound, or caused by inefficient queries. For reads, caching, appropriate indexes, and read replicas may reduce pressure depending on consistency requirements. For writes or very large datasets, batching, partitioning, or workload redesign may be appropriate. I would also check connection pools and query plans because inefficient database usage can create an avoidable bottleneck.

Why doesn't adding more application servers always improve throughput?

Additional application servers increase capacity only when the application tier is the limiting resource. If every server sends work to the same database, queue, external API, or other shared dependency, that dependency can remain the limiting factor. More application servers can even increase pressure on the shared resource and make the bottleneck worse. Scaling should therefore follow measured evidence about where capacity is constrained.

How would you design a service to handle a bottleneck caused by traffic spikes?

I would first distinguish sustained capacity limits from temporary bursts. A durable queue can absorb work when the operation is asynchronous, while rate limiting and backpressure prevent downstream resources from being overwhelmed. Caching and horizontal scaling can reduce or distribute demand where appropriate. I would also define overload behavior, monitor queue depth and latency, and ensure that recovery after the spike does not create a secondary failure cascade.