Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to know when consumer.poll() is called while using @KafkaListener annotation?

I understand that I cannot control when to poll if I use @KafkaListener, and I read from this answer that

the next poll() is performed after the last message from the previous poll has been processed by the listener.

So I'm wondering how to know when each poll() is executed? Or equivalently, how long does it take to process all messages received in each poll() call?

I am asking because my program got "Offset commit failed ... The request timed out" exceptions, and I would like to tune my consumer config, i.e. max.poll.interval.ms and max.poll.records, but I need to know the current performance first.

Here is part of my @KafkaListener method if it helps:

@KafkaListener(id = "dataListener", topics ="${spring.kafka.topic}", containerFactory = "kafkaListenerContainerFactory")
public void listen(@Payload(required = false)  ConsumerRecord payload, @Header(KafkaHeaders.RECEIVED_PARTITION_ID) String partition,
                        @Header(KafkaHeaders.OFFSET)Long offset, @Header(KafkaHeaders.RECEIVED_MESSAGE_KEY)String messageKey){
    // processing messages
}
like image 868
yifei Avatar asked Jan 25 '26 11:01

yifei


1 Answers

You can see polling activity by turning on DEBUG logging.

this.logger.debug(() -> "Received: " + records.count() + " records");

logged after each poll().

like image 152
Gary Russell Avatar answered Jan 26 '26 23:01

Gary Russell



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!