Java thottling mechanism

Question

Update: I'm on Java 1.6.34 with no chance of upgrading to Java 7.

I have a scenario where I am only allowed to call a method 80 times per minute. It's actually a service API written by a 3rd party, and it "shuts down" (ignores calls) its API if you call it too many times:

public class WidgetService {
    // Can only call this method 80x/min, otherwise it
    // it just doesn't do anything
    public void doSomething(Fizz fizz);
}

I'd like to write an ApiThrottler class that has a boolean canRun() method that will tell my Java client whether the doSomething(Fizz) method can be called or not. (Of course it can always be called, but there's no sense of calling it if we've exceeded our rate.)

So something that would allow me to write code like so:

// 80x/min
ApiThrottler throttler = new ApiThrottler(80);

WidgetService widgetService = new DefaultWidgetService();

// Massive list of Fizzes
List<Fizz> fizzes = getFizzes();

for(Fizz fizz : fizzes)
    if(throttler.canRun())
        widgetService.doSomething(fizz);

This doesn't necessarily have to be the API (ApiThrottler#canRun), but nevertheless I need a solid/reliable mechanism that will pause/sleep until WidgetService#doSomething(Fizz) can be called.

This makes me feel like we're heading into the realm of using multiple threads, which makes me feel like we could use some sort of locking mechanism and the Java notification (wait()/notify()) model. But having no experience in this realm, I can't seem to wrap my head around the most elegant solution. Thanks in advance.

Answer 1

Probably one of the best of options would be to use Semaphore http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Semaphore.html class and give it 80 permits every minute. This can be accomplished for example by using timer class http://docs.oracle.com/javase/7/docs/api/java/util/Timer.html. The caller thread will consume permits every time it performs the call to the service by calling acquire() on the semaphore, which will block if all permits have been drained already.

It would of course be possible to code this up using wait/notify and integer counter with timer or separate thread, as you mention, but that would be more complex compared to the usage of more modern java.util.concurrent API that I have outlined above.

It can look close to the following:

class Throttler implements TimerTask {
  final Semaphore s = new Semaphore(80);  
  final Timer timer = new Timer(true);

  Throttler() {
    timer.schedule(this, 0, 60*1000);   //schedule this for 1 min execution  
  }

  run() {  //called by timer 
    s.release(80 - s.availablePermits());
  }

  makeCall() {
    s.acquire();
    doMakeCall();
  }

}

This should work starting from Java 5.

Also even nicer solution would be to use com.google.common.util.concurrent.RateLimiter from Guava. It can look like this:

class Throttler {
  final RateLimiter rateLimiter = RateLimiter.create(80.0/60.0);

  makeCall() {
    rateLimiter.acquire();
    doMakeCall();
  }
}

Semantics is slightly different compared to Semaphore solution, with RateLimiter being most probably better fit in your situation.

Answer 2

I have written something like this recently. The only change is that my code expects a callback when the function is done running. So if it cant run, I directly call the callback.

One additional change is that, since this call is probably async, it is possible the call is in progress when you call it the second time. In such cases, I just ignored the call.

And my throttler has a helper function called call which takes the function to be called and the callback. This was in C++, so for you that would be in the form of Action and listener interfaces.

This has the advantage over Semaphore based solutions that it serializes requests so that you dont call it too often either.

interface Callback{
    public void OnFunctionCalled();
}

class APIThrottler
   //ctor etc
   boolean CanCall();

   public boolean IsInProgress();

   public void SetInProgress(boolean inProgress = true);

   public void Mark(){/*increment counter*/; SetInProgress(false);} // couldnt think of a better name..

   public void Call(Callable event, Callback callback){
        If(IsInProgress())
            return;
        else if(CanCall())
        {
            SetInProgress();
            event.Call();
        }
        else
            callback.OnFunctionCalled();

    }
}

Once the function callsback (or within the function itself if it is synchronous) you would need to Mark() once it is done.

This is for the most part my implementation, only difference is that I was dealing with once in x seconds (or minutes).