R plumber package for node.js parallel processing

Question

I would like to use the plumber package to carry out some flexible parallel processing and was hoping it would work within a node.js framework such that it is non-blocking...

I have the following plumber file.

# myfile.R

#* @get /mean
normalMean <- function(samples=10){
  Sys.sleep(5)
  data <- rnorm(samples)
  mean(data)
}

I have also installed pm2 as suggested here http://plumber.trestletech.com/docs/hosting/

I have also made the same run-myfile.sh file i.e.

#!/bin/bash
R -e "library(plumber); pr <- plumb('myfile.R'); pr\$run(port=4000)"

and made it executable as suggested...

I have started up pm2 using

pm2 start /path/to/run-myfile.sh

and wanted to test to see if it could carry out a non-blocking node.js framework...

by opening up another R console and running the following...

foo <- function(){
    con <- curl::curl('http://localhost:4000/mean?samples=10000',handle = curl::new_handle())
    on.exit(close(con))
    return(readLines(con, n = 1, ok = FALSE, warn = FALSE))
}

system.time(for (i in seq(5)){
    print(foo())
})

Perhaps it is my miss-understanding of how a node.js non-blocking framework is meant to work, but in my head the last loop should take only a bit of over 5 seconds. But it seems to take 25 seconds, suggesting everything is sequential rather than parallel.

How could I use the plumber package to carry out that non-blocking nature?

Answer 1

pm2 can't load-balance R processes for you, unfortunately. R is single-threaded and doesn't really have libraries that allow it to behave in asynchronous fashion like NodeJS does (yet), so there aren't many great ways to parallelize code like this in plumber today. The best option would be to run multiple plumber R back-ends and distribute traffic across them. See the "load balancing" section here: http://plumber.trestletech.com/docs/docker-advanced