Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to reinstate a Kafka Consumer which has been kicked out of the group?

I have a situation where I have a single kafka consumer which retrieves records from kafka using the poll mechanism. Sometimes this consumer gets kicked out of the consumer group due to failure to call poll within the session.timeout period which I have configured to 30s. My question is if this happens will a poll at some later point of time re-add the consumer to the group or do I need to do something else?

I am using kafka version 0.10.2.1

Edit: Aug 14 2018

Some more info. After I do a poll I never process the records in the same thread. I simply add all the records to a separate queue (serviced by a separate thread pool) for processing.

like image 692
Moiz Raja Avatar asked Oct 20 '25 14:10

Moiz Raja


1 Answers

Poll will initiate a "join group" request, if the consumer is not a member of the group yet, and will result in consumer joining the group (unless some error situation prevents it). Note that depending on the group status (other members in the group, subscribed topics in the group) the consumer may or may not get the same partitions it was consuming from before it was kicked out. This would not be the case if the consumer is the only consumer in the group.

like image 94
vahid Avatar answered Oct 23 '25 06:10

vahid