How does Apache Kafka use open file descriptors?

Question

I wanted to know how does Kafka use open file descriptors. Why is it recommended to have a large number of open file descriptor. Does it impact Producer and Consumer throughput.

Answer 1

Brokers create and maintain file handles for each log segment files and network connections. The total number could be very huge if the broker hosts many partitions and partition has many log segment files. This applies for the network connection as well.

I don't immediately see any possible performance declines caused by setting a large file-max, but the page cache miss matters.