KafkaTestUtils.getRecords() returns only first sent record

Question

Sending 4 records:

producer.send(new ProducerRecord<>("my-topic", 0, "key1", "value1"));
producer.send(new ProducerRecord<>("my-topic", 0, "key2", "value2"));
producer.send(new ProducerRecord<>("my-topic", 0, "key3", "value3"));
producer.send(new ProducerRecord<>("my-topic", 0, "key4", "value4"));

Using KafkaTestUtils.getRecords() get only first sent record;

final ConsumerRecords<String, OrderBookViewItem> records = KafkaTestUtils.getRecords(consumer, 10000);

Using consumer.poll() get all 4;

final Iterable<ConsumerRecord<String, OrderBookViewItem>> records = consumer.poll(1000).records("my-topic");

How to tune KafkaTestUtils to return all records or it is bug?

Answer 1

It simply returns whatever Kafka gives us on a poll() the method doesn't know how many records to fetch.

You can call it multiple times, or you can try setting the consumer properties fetch.min.bytes and fetch.max.wait.ms so that the poll() waits for more data.

See the kafka documentation for more information about these properties.

Answer 2

it works with 3 parameters (consumer, timeout, minRecords) for me:

final ConsumerRecords<String, OrderBookViewItem> records = KafkaTestUtils.getRecords(consumer, 10000, 4);

minRecords – wait until the timeout or at least this number of receords are