Circuit Breaker

A circuit breaker temporarily stops calls to a dependency when observed failures suggest it is unhealthy. It limits cascading failures by rejecting calls quickly, then cautiously probing for recovery.

A circuit breaker protects a service from repeatedly calling a dependency that is failing or responding too slowly. Continuing to send calls can waste local resources and add load to an already degraded service, potentially spreading the failure.

In the closed state, calls proceed and the breaker observes outcomes. When a configured failure condition is met—such as a failure ratio over a window or a consecutive-failure count—the breaker opens. While open, calls are rejected quickly or routed to a fallback rather than sent to the dependency.

After a configured delay, the breaker may enter the half-open state and allow a small number of probe calls. Successful probes indicate recovery and close the breaker; failed probes reopen it. Implementations differ in their thresholds, measurement windows, probe limits, and treatment of slow calls.

A breaker does not make the dependency healthy and does not guarantee that a failed operation will succeed later. Pair it with timeouts, sensible retry policies, observability, and a deliberate fallback or error response. In distributed systems, the breaker is commonly maintained per process and dependency, so its view may not reflect the health of every instance.

Key points

• Closed allows calls; open suppresses calls; half-open tests recovery with limited probes.
• A breaker protects callers and dependencies from repeated failing calls but does not repair the dependency.
• Use timeouts so slow or hung calls eventually produce observable outcomes.
• Keep retries bounded and avoid coordinating many clients into a retry surge.
• Choose a fallback that is correct for the operation, and monitor breaker transitions and rejected calls.

Interview questions

What is a circuit breaker, and what are its states?

A circuit breaker tracks outcomes when calling a dependency and changes how calls are handled when failures cross a configured condition. In closed state, it allows calls; in open state, it rejects them quickly or uses a fallback. After a delay, half-open mode permits limited probes. If probes succeed it closes, and if they fail it opens again.

How is a circuit breaker different from a retry policy?

A retry repeats an individual operation after an error, while a circuit breaker uses observed failures to stop making calls for a period. Retries can be useful for transient faults, but aggressive retries can increase load during an outage. A breaker limits that amplification by rejecting calls once its policy indicates the dependency is unhealthy. They can be combined with bounded retries, backoff, and jitter.

Why do circuit breakers and timeouts complement each other?

A timeout bounds how long a call can occupy a task, thread, or connection while waiting for the dependency. A breaker uses outcomes across calls to decide whether later calls should be attempted at all. Without timeouts, hung calls may not produce timely failures for the breaker to count. Both are useful: the timeout bounds an individual call, and the breaker can protect the system across calls.

How would you design the policy for opening and closing a breaker?

A policy might open when the failure ratio exceeds a threshold over a rolling window, provided enough calls have been observed. Another option is a consecutive-failure threshold; some systems also count slow calls. After an open interval, the breaker allows a small number of probes rather than releasing all traffic at once. The exact policy should reflect traffic volume and the dependency's recovery characteristics.

What should a service do when the breaker rejects a call?

I would first identify which operations depend on the failing service and what a useful degraded response can be. Depending on the operation, the fallback could be cached data, a partial result, or a clear error; returning fabricated success can be unsafe. I would make the fallback's freshness and limitations visible where relevant. I would also measure rejected calls and fallback outcomes so degraded behavior is observable.

Should breaker state be local to a process or shared across instances?

A process-local breaker is simple and reacts to that instance's own observations, but each instance may make a different state transition. A shared breaker can provide a broader view, but adds coordination, latency, and another dependency whose failure must be handled. The right choice depends on whether coordinated suppression is worth that complexity. Regardless, breaker state should be observable and should not be mistaken for an authoritative health check.