Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Node.js: Efficiently reading a range of lines

Tags:

node.js

I'm currently using Node.js and am wondering how one would read a range of lines from a large text file. An obvious solution would be like so:

var fs = require('fs');
fs.readFile(file, function(err, data) {
  var lines = data.split('\n');
});

However, that would involve loading the entire file into memory, which would be impractical for large text files, such as ones 100MB+.

In Bash, I would normally use sed for this case.

like image 959
hexacyanide Avatar asked Sep 03 '25 09:09

hexacyanide


1 Answers

With lazy:

var fs   = require('fs'),
    lazy = require('lazy');

var x = 23;
var y = 42;
var lines = (
   lazy(fs.createReadStream('./large.txt'))
     .lines
     .skip(x - 1)
     .take(y - x + 1)
);
lines.forEach(function(line) {
    console.log(line.toString('utf-8'));
});
like image 121
phihag Avatar answered Sep 05 '25 01:09

phihag