Programmatically reduce log in a spark shell

Question

Can I programmatically reduce the log in a spark shell by removing all the "INFO" flags? It is spamming my window and I can't analyze the real output For example:

15/01/09 12:23:02 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 649 bytes result sent to driver
15/01/09 12:23:02 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 500 ms on localhost (1/1)
15/01/09 12:23:02 INFO DAGScheduler: Stage 0 (count at MainApp.scala:31) finished in 0.520 s
15/01/09 12:23:02 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 
15/01/09 12:23:02 INFO DAGScheduler: Job 0 finished: count at MainApp.scala:31, took 0.639191 s

Looking for any alternatives if you have some!

Answer 1

This removes most info messages (not all though)

import org.apache.log4j.{Level, Logger}
// ...
val level = Level.WARN
Logger.getLogger("org").setLevel(level)
Logger.getLogger("akka").setLevel(level)

Or as util methods:

def setLogLevel(level: String): Unit = {
  setLogLevel(Level.toLevel(level, Level.INFO))
}

def setLogLevel(level: Level): Unit = {
  Logger.getLogger("org").setLevel(level)
  Logger.getLogger("akka").setLevel(level)
}