increasing NumPy memory limit

Question

I am currently doing coding some NN for huge dataset, for example MNIST dataset (about 700*50000). But when I test it, my code got MemoryError. I have a computer with 12 GB ram, but I think Python or Numpy can't use all of them.

Can I push Python or Numpy to use all remaining available memory in my PC ?

OS : Windows 7 64-bit

Python : Python(x, y) 2.7.60

Thanks

Answer 1

I believe that the Python(x, y) distribution of Python is still only a 32-bit build (64-bit support is still on its roadmap), so you are limited to 32 bits of address space even though you are using a 64-bit OS. You will need to install a 64-bit build of Python and numpy binaries to get access to more memory.