IO/Network-bound Fork/Join-Tasks

Question

I am using an unbounded, thread-caching Java-ExecutorService which concurrently processes my tasks. Now I have some tasks that do just some lengthy operations (the minority) and some tasks that do blocking IO (connecting to an URL, the majority). If running my application and starting one workload, about 70 tasks are dispatched to the executor which fires up 70 threads and processes all of them concurrently.

I want to limit the threads per host that the task wants to connect to and if possible, also the number of overall tasks. My problem is that all tasks do something like fork-join, so they spawn other tasks and wait for their result before proceeding. This means I can't just use a semaphore or a bounded-queue per host, since a tasks could fork other tasks which won't get processed until the semaphore/queue has a free slot, thus the application is deadlocked.

Is there some pattern to approach this problem?

Refined: IO(=network)-bound Tasks are submitted. After doing some IO-work they potentially fork other IO-Bound tasks, then join them and, potentially, do additional IO-bound processing.

This seems to be Fork/Join with IO-bound Tasks. How do I limit/throttle the execution of IO-performing tasks without deadlocking the thread that initially forked the tasks (deadlocked because its forked child-tasks won't get processed due to limits/throttling) ?

Thanks a bunch

Answer 1

Async is definitely the way to go, as they compose very nicely. While the programming model is painful in Java, using Finagle on top of Netty is a breeze with Scala, where all operations are basically a function that takes a Request and returns a Future[Response] that can be mapped (transformed) across.

See also:

• http://twitter.github.com/scala_school/finagle.html
• https://github.com/twitter/finagle