Which ExecutorService is best for blocking IO tasks

Question

Let's imagine that we have n independent blocking IO tasks e.g. tasks for rest-call to another server. Then all answer we need to combine. Every task can be processing over 10 second.

1. We can process it sequentially and spent ~n*10 second at the end:

   Task1Ans task1 = service1.doSomething();
   Task2Ans task2 = service2.doSomething()
   ...
   return result;
   

2. Another strategy is to process it in parallel manner using CompletableFuture and spent ~ 10 second on all task:

   CompletableFuture<Task1Ans> task1Cs = CompletableFuture.supplyAsync(() -> service1.doSomething(), bestExecutor);
   CompletableFuture<Task2Ans> task2Cs = CompletableFuture.supplyAsync(() -> service2.doSomething(), bestExecutor);
   return CompletableFuture.allOf(task1Cs, task2Cs)
      .thenApply(nothing -> {
          ...
          // combine task1, task2 into result object
          return result;
      }).join();
   

The second approach has benefits, but I can't understand which type of thread pool is the best for this kind of task:

ExecutorService bestExecutor = Executors.newFixedThreadPool(30)   /// or Executors.newCachedThreadPool() or Executors.newWorkStealingPool()

My question is which ExecutorService is best for process n-parallel blocking IO tasks.

Answer 1

On completely CPU bound tasks you do not get additional performances by going with more threads than CPU cores. So in this scenario, 8 core / 8 thread CPU needs only 8 thread to maximize performances, and loses performance by going with more. IO tasks usually do gain performances by going with larger number of threads than CPU cores, as CPU time is available to do other stuff while waiting for IO. But even when CPU overhead of each thread is low there are limits to scaling as each thread eats into memory, and incurs caching/context switches..

Given that your task is IO limited, and you didn't provide any other constraints, you should probably just run different thread for each of your IO tasks. You can achieve this by either using fixed or cached thread pool.

If the number of your IO tasks is very large (thousands+), you should limit the maximum size of your thread pool, as you can have such thing as too many of threads.

If your task are CPU bound, you should again limit thread pool to even smaller size. Number of cores can be dynamically fetched by using:

int cores = Runtime.getRuntime().availableProcessors();

Also, just as your CPU has scaling limit, your IO device usually has a scaling limit too. You should not exceed that limit, but without measuring it is hard to say where limit is.