Reading a large text file in parallel in C++

Question

I have a large textfile.. I want to read this file and perform some manipulation in it..

This manipulation occurs independently on each line. So basically, I am looking for some function which can do this parallel.

void readFile(string filename){

  //do manipulation

}

That do manipulation can happen in parallel.

Agreed that this can be done easily using hadoop but that is an overkill solution. (Its large file but not that large that I need hadoop for this...)

How do I do this in C++?

Answer 1

I would use mmap for that. mmap gives you memory-like access to file so you can easly read in parallel. Please look at another stackoverflow topic about mmap. Be careful when usin non-read-only pattern with mmap.