How to alloc more than MaxInteger bytes of memory in C#

Question

I wish to allocate more than MaxInteger bytes of memory.

Marshall.AllocHGlobal() expects an integer - so I cannot use this. Is there another way?

Update

I changed the platform to x64, and then I ran the code below.

myp appears to have the right length: about 3.0G. But stubbornly "buffer" maxes out at 2.1G.

Any idea why?

    var fileStream = new FileStream(
          "C:\\big.BC2",
          FileMode.Open,
          FileAccess.Read,
          FileShare.Read,
          16 * 1024,
          FileOptions.SequentialScan);
    Int64 length = fileStream.Length;
    Console.WriteLine(length);
    Console.WriteLine(Int64.MaxValue);
    IntPtr myp = new IntPtr(length);
    //IntPtr buffer = Marshal.AllocHGlobal(myp);
    IntPtr buffer = VirtualAllocEx(
        Process.GetCurrentProcess().Handle,
        IntPtr.Zero,
        new IntPtr(length),
        AllocationType.Commit | AllocationType.Reserve,
        MemoryProtection.ReadWrite);
    unsafe
    {
        byte* pBytes = (byte*)myp.ToPointer();
        var memoryStream = new UnmanagedMemoryStream(pBytes, (long)length, (long)length, FileAccess.ReadWrite);
        fileStream.CopyTo(memoryStream);

Answer 1

That's not possible on current mainstream hardware. Memory buffers are restricted to 2 gigabytes, even on 64-bit machines. Indexed addressing of the buffer is still done with a 32-bit signed offset. It is technically possible to generate machine code that can index more, using a register to store the offset, but that's expensive and slows down all array indexing, even for the ones that aren't larger than 2 GB.

Furthermore, you can't get a buffer larger than about 650MB out of the address space available to a 32-bit process. There aren't enough contiguous memory pages available because virtual memory contains both code and data at various addresses.

Companies like IBM and Sun sell hardware that can do this.