Write results to a file inside foreach loop R

Question

I am using the foreach package with %dopar% to implement a set of simulations. The structure of the simulation is illustrated by this simple version:

library(foreach)
library(doMC)
registerDoMC(4) 

id.list <- c(1:3000)

results <- foreach(i=1:1000,.combine=rbind) %dopar% {

    ## Randomly draw 100 donor pool units and 1 treated unit
    v1 <- sample(id.list, 1, replace = FALSE)  
    v2 <- sample(id.list, 1, replace = FALSE)
    v3 <- sample(id.list, 1, replace = FALSE)
    v4 <- sample(id.list, 1, replace = FALSE)

    c(i,v1,v2,v3,v4)
}

I would like to write the results from this loop to a .csv file or something similar while the loop is still running. For instance, when the loop hits iteration 500, I'd like to write the first 500 rows to .csv.

Alternatively, I would be open to breaking the loop when it hits iteration 500, while extracting the first 500 rows of results.

Is anything like this possible within foreach?

Answer 1

say, instead of 1000 trys, you were going for 10000, I would break it up like this, using part_num to differentiate the blocks of size 500

library(foreach)
library(doMC)
registerDoMC(4) 

id.list <- c(1:3000)

for(part_num in 1:20){
  results <- foreach(i=1:500,.combine=rbind) %dopar% {
    ## Randomly draw 100 donor pool units and 1 treated unit
    v1 <- sample(id.list, 1, replace = FALSE)  
    v2 <- sample(id.list, 1, replace = FALSE)
    v3 <- sample(id.list, 1, replace = FALSE)
    v4 <- sample(id.list, 1, replace = FALSE)
    c(i,v1,v2,v3,v4)
  }

  write.csv(results, file = paste('part', part_num, '.csv', sep = ''))
}