Kafka Producer Metrics

Question

I am running a Kafka producer in a local machine using my Intellij IDE & the producer will be producing a million records. While doing so, I want to capture the producer metrics in the below way:

I am aware about JMX port for kafka & I did try setting the Kafka JMX port to 9999. But I am not sure if we can get the metrics using JConsole or JVisualVM in the above way that I am expecting.

Can any one suggest any idea as to how can this be achieved?

Answer 1

In addition of JMX, the official Kafka clients also expose their metrics via the Java API, see the metrics() method to retrieve them all.

For example, to print all the metrics' names and values:

for (Entry<MetricName, ? extends Metric> entry : producer.metrics().entrySet()) {
    System.out.println(entry.getKey().name() + " : " + entry.getValue().metricValue());
}

Out of all the metrics, you are probably interested in outgoing-byte-rate, request-total and request-rate.