Running Storm nimbus and supervisor on the same physical node in cluster mode

Question

I've a storm cluster of 2 physical nodes right now. I'm running storm nimbus on node-1 and storm supervisor on node-2. Looks like all my topologies are running on running on node-2 (supervisor node) only. Should I run supervisor on node-1 as well ?

Thanks

Answer 1

You could, but I wouldn't recommend it.

In Storm's current design, nimbus is a single point of failure (there's plans to address this), but running a supervisor on the same node as nimbus makes it more likely that something bad might happen to the nimbus node, which would be catastrophic for your Storm cluster.

Further, part of Storm's design is that the workers and the supervisor nodes should be able to die and Storm should be able to recover. If you use your node-1 as a supervisor in addition to it being the nimbus server, you lose some of that flexibility.

Finally, as your cluster grows, your nimbus server will have plenty to do on its own and you want it to operate quickly so it doesn't slow down your workers since it could be a bottleneck if you don't give it adequate resources.

Answer 2

If you want topologies to run on node-1, then yes, you should run the Supervisor process on node-1 as well. The Nimbus helps to coordinate work among Supervisors, but does not execute a topology's Workers itself. For more details, see http://storm.incubator.apache.org/documentation/Tutorial.html