Parse output of spawned node.js child process line by line

Question

I have a PhantomJS/CasperJS script which I'm running from within a node.js script using process.spawn(). Since CasperJS doesn't support require()ing modules, I'm trying to print commands from CasperJS to stdout and then read them in from my node.js script using spawn.stdout.on('data', function(data) {}); in order to do things like add objects to redis/mongoose (convoluted, yes, but seems more straightforward than setting up a web service for this...) The CasperJS script executes a series of commands and creates, say, 20 screenshots which need to be added to my database.

However, I can't figure out how to break the data variable (a Buffer?) into lines... I've tried converting it to a string and then doing a replace, I've tried doing spawn.stdout.setEncoding('utf8'); but nothing seems to work...

Here is what I have right now

var spawn = require('child_process').spawn;  var bin = "casperjs" //googlelinks.js is the example given at http://casperjs.org/#quickstart var args = ['scripts/googlelinks.js']; var cspr = spawn(bin, args);  //cspr.stdout.setEncoding('utf8'); cspr.stdout.on('data', function (data) {     var buff = new Buffer(data);     console.log("foo: " + buff.toString('utf8')); });  cspr.stderr.on('data', function (data) {     data += '';     console.log(data.replace("\n", "\nstderr: ")); });  cspr.on('exit', function (code) {     console.log('child process exited with code ' + code);     process.exit(code); }); 

https://gist.github.com/2131204

Answer 1

Try this:

cspr.stdout.setEncoding('utf8'); cspr.stdout.on('data', function(data) {   var str = data.toString(), lines = str.split(/(\r?\n)/g);   for (var i=0; i<lines.length; i++) {     // Process the line, noting it might be incomplete.   } }); 

Note that the "data" event might not necessarily break evenly between lines of output, so a single line might span multiple data events.

Answer 2

I've actually written a Node library for exactly this purpose, it's called stream-splitter and you can find it on Github: samcday/stream-splitter.

The library provides a special Stream you can pipe your casper stdout into, along with a delimiter (in your case, \n), and it will emit neat token events, one for each line it has split out from the input Stream. The internal implementation for this is very simple, and delegates most of the magic to substack/node-buffers which means there's no unnecessary Buffer allocations/copies.