Using Smart Pointers in Cython for Dynamically Allocated Arrays

Question

I am writing a Python wrapper for a function call with the signature

double** foo(double** arrayIn, int dim1, int dim2);

and need to build arrayIn inside my Python wrapper. One possible solution for this is given here. However, since Cython includes support for smart pointers, I would prefer to implement that solution. One way to do this would be a combination of malloc and a unique_ptr with a custom deleter. Another (simpler) solution would be to use the allocator class from libcpp.

import numpy as np
cimport numpy as np
from libcpp.memory cimport unique_ptr, allocator

def testArray(int dim1, int dim2):
    cdef allocator[double *] ptr_al
    cdef unique_ptr[double *] myptr
    cdef np.ndarray arr
    cdef double[:,:] carr

    myptr.reset(ptr_al.allocate(dim1))
    arr = np.ndarray((dim1,dim2),dtype=np.float64,order='C')
    carr = arr

    myptr.get()[0] = &carr[0,0]
    myptr.get()[1] = &carr[1,0]
    myptr.get()[2] = &carr[2,0]

This code compiles and executes correctly (with Cython 24.1, Python 3.5, VS2015). My concern is whether or not everything will be properly deallocated / garbage collected. My understanding is that Python is responsible for the ndarray, and unique_ptr should be responsible for the double *[] created by allocator. Is this correct, or will the code create a memory leak? Is there a way I could verify that everything has been properly deallocated?

Answer 1

Is this correct, or will the code create a memory leak?

I don't think this should leak.

Is there a way I could verify that everything has been properly deallocated?

You can call testArray in a loop and see if the process memory is growing linearly or stays about constant. Since you know dim1 and dim2, you can estimate the memory leak size if something isn't freed properly.

There are other ways to test for memory leaks, in more complicated situations: there are debug versions of the C library that will tell you if you've deallocated all of the memory you allocated. Also, there are tools like valgrind, and clang's leaksanitizer, but in your case I'd go with the loop.