What is the most efficient way to read formatted data from a large file?

Question

Options: 1. Reading the whole file into one huge buffer and parsing it afterwards. 2. Mapping the file to virtual memory. 3. Reading the file in chunks and parsing them one by one.

The file can contain quite arbitrary data but it's mostly numbers, values, strings and so on formatted in certain ways (commas, brackets, quotations, etc). Which option would give me greatest overall performance?

Answer 1

If the file is very large, then you might consider using multiple threads with option 2 or 3. Each thread can handle a single chunk of file/memory and you can overlap IO and computation (parsing) this way.