Consumer Group Offset Reset

A Kafka consumer group’s offsets determine where its partitions resume consumption. Resetting offsets moves those positions backward or forward, allowing controlled replay or skipping of records.

Kafka stores a committed offset for each consumer-group partition. When a consumer restarts or a group rebalances, the consumer uses the committed position to determine where to continue reading.

An offset reset changes the group’s stored position, typically to a chosen offset or to a policy such as the earliest or latest available record. Moving offsets backward can replay previously consumed events; moving them forward can skip records.

For event replay, a common approach is to stop the consumers, reset the group offsets, and then restart the consumers. The reset must target the intended topic partitions and consumer group so that unrelated consumers are not affected.

Offset resets do not recreate deleted Kafka records. If the requested position is outside the available retention range, Kafka can only start from an available offset according to the applicable reset behavior.

Key points

• Offsets are tracked per consumer group and partition.
• Moving offsets backward enables replay of retained records.
• Moving offsets forward can skip records that have not been processed from the new position.
• Consumers should generally be stopped before performing an administrative offset reset.
• Resetting an offset does not restore records already removed by retention.

Interview questions

What happens if the reset target is older than Kafka's retained data?

Kafka cannot consume records that have already been removed by retention or other deletion policies. The earliest available offset may therefore be newer than the desired replay position. If a reset requests an unavailable position, the effective starting position depends on the operation and configured reset behavior. Before planning a replay, I would verify that the required records are still retained.

What is a Kafka consumer offset?

An offset identifies a position within a Kafka partition. A consumer group commits offsets so Kafka knows where that group should resume consumption. Each partition has its own offset sequence, so a group can have different positions across partitions. The committed position is therefore a piece of consumer-group state rather than a property of the message itself.

Why would you reset consumer-group offsets?

The most common reason is to control where consumption resumes. Moving offsets backward can replay retained events after a bug fix or when rebuilding derived data. Moving them forward can intentionally skip records that are no longer relevant. The reset should be scoped carefully to the correct group and partitions.

How would you safely perform an offset reset for a production consumer group?

First, identify the exact consumer group, topics, and partitions involved and verify the desired target offsets. I would normally stop or otherwise quiesce the consumers before changing their committed positions, then perform the reset and verify the resulting offsets. After restarting consumption, I would monitor lag, processing errors, and downstream effects. The procedure should also account for duplicate processing because replayed records may be processed again.