Concurrently and asynchronously write into a file with MSVC?

Question

Currently I am using a single thread and fseek/fwrite to save data into a large file in loop. But of course the saving is bottleneck. To remove this bottleneck, I think I can create a thread for asynchronous file write but this will block the file access in next iteration thread.

Is there any way to concurrently and asynchronously write into a file in Visual C/C++?

(OS is Windows)

Answer 1

Use the function:

CreateFileMapping

http://msdn.microsoft.com/en-us/library/windows/desktop/aa366537(v=vs.85).aspx

to memory map the output file to memory, and then you can have your multiple threads write to the memory concurrently.

More info about memory mapped files here:

http://en.wikipedia.org/wiki/Memory-mapped_file

Answer 2

Use CreateFile(), SetFilePointer(), and SetEndOfFile() to create and pre-size a file. Then use CreateFile() in your worker threads to open additional handles to the file, SetFilePointer() to seek to the desired offsets, and WriteFile() to write into it. As long as each call to CreateFile() grants FILE_SHARE_WRITE rights, you can open multiple handles with GENERIC_WRITE permissions at a time. Each handle maintains its own pointer in the file.