I would like to reserve a region of memory and later map files contiguously into the reserved memory. There may be large time gaps between mapping the files during which other functions may allocate memory from the heap. Once mapped, a file may not be unmapped and mapped to a new memory location.
On Linux that would be something like:
#include <iostream>
#include <sys/mman.h>
#include <unistd.h>
#include <fcntl.h>
#include <cerrno>
int main(){
void *memory = mmap(nullptr, getpagesize() * 2,
PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE, -1, 0); // reserve memory
int handle1 = ::open("1", O_CREAT | O_RDWR, S_IRWXU); // open file1
int handle2 = ::open("2", O_CREAT | O_RDWR, S_IRWXU); // open file2
void *data = mmap(memory, getpagesize(), PROT_READ | PROT_WRITE, MAP_SHARED |MAP_FIXED, handle1, 0); // map first file into reserved memory
void *data2 = mmap(static_cast<char *>(memory) + getpagesize(), getpagesize(), PROT_READ | PROT_WRITE, MAP_SHARED | MAP_FIXED, handle2, 0); // map second file into reserved memory
}
On Windows however I can't seem to find the correct way to do this. Does anyone know how this is done?
Edit: Since the my goal seems to be not that easy to understand. Once again simplified:
I would like to have memory that is memory mapped. Once the mapped memory is filled with data I would like to map a new file into memory directly after the already mapped memory to extend the mapped memory (without leaving a gap between the two memory mapped regions).
At a later program start the files may be used to restore all the data of the previous run.
Problem with solution with sparse files and extending existing file: Program should be able to remove files if they aren't needed any more. Therefore its important that the mappings are always new files.
You may think of it more as an allocator. Memory mapped memory is needed. The library maps a block of memory and returns a pointer to a subblock. The memory is no longer needed, it is returned to the allocator. In case an entire mapping is no longer needed, the associated file is removed (The data does not need to be written by the mapping if it is no longer needed).
You don't really need to have it reserved when you map in the files, you just need to know that you can map in the two files contiguously somewhere. For example your Linux example code, being single threaded, would work just as well if you immediately unmapped the reserved region before mapping in the files.
On Windows, handling the possible multithreaded race condition, you can do something like:
while(1) {
char *memory = VirtualAlloc(NULL, page_size * 2, MEM_RESERVE, 0);
VirtualFree(memory);
if (MapViewOfFileEx(handle1, FILE_MAP_WRITE, 0, 0, page_size, memory) == NULL
&& GetLastError() == ERROR_INVALID_ADDRESS) {
continue;
}
if (MapViewOfFileEx(handle2, FILE_MAP_WRITE, 0, 0, page_size, memory + page_size) == NULL
&& GetLastError() == ERROR_INVALID_ADDRESS) {
UnMapViewOfFile(memory);
continue;
}
break;
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With