Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

"un-register" a doParallel cluster

If I run foreach... %dopar% without registering a cluster, foreach raises a warning, and executes the code sequentially:

library("doParallel") foreach(i=1:3) %dopar%   sqrt(i) 

Yields:

Warning message: executing %dopar% sequentially: no parallel backend registered  

However, if I run this same code after starting, registering, and stopping a cluster, it fails:

cl <- makeCluster(2) registerDoParallel(cl) stopCluster(cl) rm(cl) foreach(i=1:3) %dopar%   sqrt(i) 

Yields:

Error in summary.connection(connection) : invalid connection 

Is there an opposite of registerDoParallel() that cleans up the cluster registration? Or am I stuck with the ghost of the old cluster until I re-start my R session?

/edit: some googling reveals the bumphunter:::foreachCleanup() function in the bumphunter Biocondoctor package:

function ()  {     if (exists(".revoDoParCluster", where = doParallel:::.options)) {         if (!is.null(doParallel:::.options$.revoDoParCluster))              stopCluster(doParallel:::.options$.revoDoParCluster)         remove(".revoDoParCluster", envir = doParallel:::.options)     } } <environment: namespace:bumphunter> 

However, this function doesn't seem to fix the problem.

library(bumphunter) cl <- makeCluster(2) registerDoParallel(cl) stopCluster(cl) rm(cl) bumphunter:::foreachCleanup() foreach(i=1:3) %dopar%   sqrt(i) 

Where does foreach keep the information on the registered cluster?

like image 544
Zach Avatar asked Aug 02 '14 17:08

Zach


2 Answers

The only official way to "unregister" a foreach backend is to register the sequential backend:

registerDoSEQ() 

This makes sense to me because you're supposed to declare which backend to use, so I didn't see any point in providing a way to "undeclare" which backend to use. Instead, you declare that you want to use the sequential backend, which is the default.

I originally considered including an "unregister" function, but since I couldn't convince myself that it was useful, I decided to leave it out since it's much easier to add a function than to remove one.

That being said, I think all you need to do is to remove all of the variables from foreach:::.foreachGlobals which is where foreach keeps all of its state:

unregister <- function() {   env <- foreach:::.foreachGlobals   rm(list=ls(name=env), pos=env) } 

After calling this function, any parallel backend will be deregistered and the warning will be issued again if %dopar% is called.

like image 187
Steve Weston Avatar answered Sep 28 '22 04:09

Steve Weston


    cl <- makeCluster(2)     registerDoParallel(cl)     on.exit(stopCluster(cl)) 

This worked fine for me.

like image 22
Smit Avatar answered Sep 28 '22 03:09

Smit