Asymmetric benchmarks in JMH

Question

I'm running performance tests on a custom collection implementation with JMH.

I'd like to imitate a scenario, where number of reads is 10x bigger than number of writes.

I used this asymmetric benchmark example and created a group with 10 reader threads and 1 writer thread:

@Benchmark
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Group("g0")
@GroupThreads(1)       
public void baselinePut0(CacheState0 state) { writing }

@Benchmark
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Group("g0")
@GroupThreads(10)
public Integer baselineGet0(CacheState0 state) { reading }

I run the test with -wi 10 -i 10 -f 1 params. In a report, variable cnt is same for all benchmarks:

Benchmark                  Mode  Cnt     Score      Error  Units
Benchmark.g0               avgt   10   262,537 ?  215,406  us/op
Benchmark.g0:baselineGet0  avgt   10     2,101 ?    0,154  us/op
Benchmark.g0:baselinePut0  avgt   10  1252,231 ?  697,807  us/op

Does it mean that number of reads was equal to number of writes in the experiment? If so, how to implement it correctly? And more general: am I missing something in this setup?

Answer 1

Cnt shows number of samples (not number of threads). In your case it's 10, since you are running your test with -i 10. It will be easier to see that this parameter is not number of threads if you run it with all params set to unique values (e.g. -i 13, while @GroupThreads(10) remains the same)

You can also (temporary) add the output line to your test and see where each thread is coming from, e.g. for reader (and similar for writer with the word "writer"):

System.out.println("reader " + Thread.currentThread().getName());