Node.js piping gzipped streams in CSV module

Question

I have a gzipped CSV file that I would like to read, perform some transformations, and write back somewhere gzipped. I am using the node-csv module for CSV transformations.

An simplified version of the code looks like this:

// dependencies
var fs = require('fs'),
    zlib = require('zlib'),
    csv = require('csv');  // http://www.adaltas.com/projects/node-csv/

// filenames
var sourceFileName = process.argv[2] || 'foo.csv.gz';
    targetFileName = process.argv[3] || 'bar.csv.gz';

// streams
var reader = fs.createReadStream(sourceFileName),
    writer = fs.createWriteStream(__dirname + '\\' + targetFileName),
    gunzip = zlib.createGunzip(),
    gzip = zlib.createGzip();

csv()
    .from.stream( reader.pipe(gunzip) )
    .to.stream( gzip.pipe(writer) )  // <-- the output stream
    .transform( function(row) {
      // some operation here
      return row;
    });

The problem is that this codes effectively writes a file with the specified name, although not gzipped, i.e. if the file gets the .gz removed, it can be opened as a regular CSV.

The question then is, how can the csv().to.stream() be passed an output stream that gzips the data and pipes it to a writer?

Thanks!

Answer 1

You're pipeing the csv to the writer because .pipe returns it's argument for chaining.

You need to change:

    .to.stream( gzip.pipe(writer) )  // <-- the output stream 

To:

    .to.stream( gzip )  // <-- the output stream  
 . . . 
gzip.pipe(writer);