Why is valgrind limited to 32 Gb on 64 bit architectures?

Question

I'm trying to valgrind a heisenbug on a process which uses more than 32 Gb of ram and hitting this valgrind limitation, is this an arbitrary one that could be avoided by recompiling valgrind or is it a hard one?

Answer 1

The limit is arbitrary and can be changed by recompiling valgrind.

Valgrind keeps track of memory using a 2-level sparse array. 16 bits of the address are used to index into a 65536-entry table containing a pointer to a second-level map, which tracks a range of the address space (normally 2¹⁶ bytes = 64KB for 32-bit processes, 2¹⁹ bytes = 512KB for 64-bit processes). So the total amount of memory that can be tracked by this 2-level sparse array is 65536 × this chunk size. By modifying the code, the size of these chunks can be increased to a larger power of 2, at the cost of using more memory to track most partial chunks.

In this message on the valgrind-users mailing list, Julian Seward explains how to increase the limit from 32GB to 128GB:

• in memcheck/mc_main.c change N_PRIMARY_BITS from 19 to 21

• change the assertions at the end of memcheck/mc_main.c accordingly:

  • MAX_PRIMARY_ADDRESS → 4 * (existing value + 1) - 1
  • for the MASK(1/2/4/8) assertions, set to zero the rightmost two '1' bits in the block of leading 1 bits, eg MASK(8) → 0xFFFFFFE000000007ULL I think
    
    

• in coregrind/m_aspacemgr/aspacemgr-linux.c change aspacem_maxAddr from (Addr)0x800000000 - 1 to (Addr)0x2000000000ULL - 1.

---

Update: The maximum amount of memory that Valgrind can use has increased in recent versions:

• 32 GB prior to Valgrind 3.9
• 64 GB in Valgrind 3.9 through 3.12
• 128 GB in Valgrind 3.13