Graceful Shutdown: In-Flight Request Completion

Graceful shutdown should stop accepting new work while allowing active requests and required background operations to finish safely. The key is to track in-flight work and wait for it within a bounded shutdown deadline.

During graceful shutdown, an application first prevents new work from entering the service, then gives work already in progress an opportunity to complete. For a Java backend, this commonly means coordinating the HTTP server, request handlers, thread pools, database work, and other managed executors.

An in-flight request is work that has started but has not completed yet. A shutdown coordinator can maintain an active-work count or use lifecycle-aware executors to determine when the application is quiescent. New requests should be rejected or prevented once shutdown begins, while existing requests continue until they complete or the deadline expires.

Waiting indefinitely is unsafe because a request can hang due to a downstream dependency, deadlock, or other failure. A graceful shutdown therefore normally has a timeout. When the deadline is reached, the application may interrupt or forcefully terminate remaining work, depending on the framework and operation involved.

Correct shutdown also requires ordering. For example, stopping request admission before closing a database connection pool helps prevent newly started requests from racing with resource teardown. The exact ordering depends on the application's dependencies, but the general rule is to stop producers before destroying resources consumed by active work.

Key points

• Stop accepting new work before tearing down shared resources.
• Track active requests and other shutdown-relevant background work.
• Wait for in-flight work only up to a bounded deadline.
• Order component shutdown according to their dependencies.
• Have a forced-shutdown path for work that does not finish.

Interview questions

What does graceful shutdown mean for a Java backend service?

It means the service stops accepting new requests while allowing requests already in progress to finish. The application waits for that work for a limited amount of time. After the deadline, remaining work may be interrupted or the process may be terminated. Shared resources should be closed only after dependent work has stopped using them.

How would you track in-flight requests during shutdown?

One approach is to increment a counter when a request is admitted and decrement it in a guaranteed completion path such as `finally`. Shutdown first disables admission and then waits for the counter to reach zero. The admission operation must avoid a race where shutdown begins after the initial acceptance check but before the counter is incremented. In production, framework lifecycle facilities can often provide safer integration than implementing this protocol around every request manually.

Why does graceful shutdown need a timeout?

An active request can fail to complete because of a stuck dependency, deadlock, network problem, or application bug. Waiting forever would prevent the service from terminating and could interfere with deployments or failover. A shutdown deadline provides a bounded recovery path. After the deadline, the application can escalate to cancellation or forced termination according to its operational requirements.

What shutdown ordering problems can cause production failures?

A common failure is closing a dependency while active requests still require it. For example, closing a database pool before request processing has drained can turn otherwise successful requests into failures. The service should first stop admission, allow dependent work to drain, and then close the resources that work consumes. The exact sequence should follow the application's dependency graph rather than a generic list of components.

How would you test graceful shutdown under concurrent load?

I would create requests with controlled completion times and initiate shutdown while they are active. The test should verify that new requests are rejected after shutdown begins, active requests can finish within the configured deadline, and resources remain available until those requests complete. I would also test a request that exceeds the deadline to verify the escalation path. Finally, I would inspect logs and metrics for dropped work, failed requests, and shutdown duration.