Per-Tenant Rate Limiting

Per-tenant rate limiting controls resource consumption independently for each tenant. It prevents one customer from consuming a disproportionate share of shared application capacity while allowing different tenants to have different limits.

In a multi-tenant service, a global rate limit treats all requests as one pool. Per-tenant rate limiting instead identifies the tenant associated with each request and maintains a separate allowance or counter for that tenant. A tenant might be an organization, customer account, or subscription.

A limiter commonly uses algorithms such as token bucket or sliding window. With token bucket, each tenant has a bucket that refills at a configured rate and loses tokens when requests are accepted. When the bucket cannot satisfy a request, the service can reject it with HTTP `429 Too Many Requests` or apply another backpressure policy.

In a distributed Java backend, the limiter state usually cannot live only in one application instance because requests for the same tenant may reach different instances. A shared store such as Redis can hold counters or bucket state, while an atomic operation or server-side script prevents concurrent requests from bypassing the limit.

Limits can be configured by tenant plan or policy, such as different sustained rates and burst capacities. The tenant identifier must be established from trusted authentication or authorization context rather than an arbitrary request parameter, otherwise a caller could evade the intended quota by changing the identifier.

Rate limiting protects short-term request capacity; quotas often describe longer-term consumption such as monthly API calls. A system may use both: a per-tenant rate limiter for immediate traffic control and a quota mechanism for aggregate usage.

Key points

• Maintain independent rate-limit state for each tenant rather than one global bucket.
• Use a defined algorithm such as token bucket and distinguish sustained rate from burst capacity.
• Distributed instances need shared or consistently coordinated limiter state.
• Derive the tenant identity from trusted authentication or authorization context.
• Return `429 Too Many Requests` when requests exceed the applicable limit, when that is the chosen API contract.

Interview questions

What is the difference between token bucket and a fixed-window counter for tenant rate limiting?

A fixed-window counter counts requests within discrete time intervals, which makes it simple but can permit bursts around a window boundary. Token bucket models a refill rate and a maximum token capacity, allowing controlled bursts while preserving an average rate. Token bucket therefore gives more direct control over sustained traffic and burst size. The appropriate choice depends on the traffic pattern and operational requirements.

Why would you use per-tenant rate limiting instead of a single global rate limiter?

A global limiter can allow a busy tenant to consume most of the shared request capacity. Per-tenant limiting gives each tenant an independent allowance, which improves isolation. It also allows limits to reflect different subscription plans or contractual policies. A global limiter can still be useful as an additional protection for the entire service.

How would you implement per-tenant rate limiting in a distributed Java service?

I would first derive the tenant ID from trusted authentication or authorization context. I would then keep limiter state in a shared system such as Redis so requests routed to different application instances see consistent state. An algorithm such as token bucket can represent both sustained throughput and bursts. The state update must be atomic so concurrent requests cannot all observe the same available capacity and exceed the limit.

How would you prevent a client from bypassing a per-tenant limiter?

The limiter must use a tenant identity established by trusted authentication and authorization, not a freely editable request field. I would ensure that the authenticated principal is authorized for the tenant represented in the request. The limiter key should then be derived consistently from that trusted identity. I would also consider other dimensions, such as API key or endpoint, when the threat model requires them.

What should happen when the rate limit is exceeded, and what should the API communicate to clients?

A common API behavior is to reject the request with HTTP `429 Too Many Requests`. The response can communicate retry information so well-behaved clients can back off rather than immediately retrying. The service should apply the policy consistently and avoid turning retries into another source of overload. Logging and metrics should distinguish rejected requests by tenant and relevant limit.