Memory leak when calling C.malloc/C.free in goroutines

Question

I'm seeing a constant memory leak when I look at RSS with the following trivial program:

package main

/*
#include <stdlib.h>
*/
import "C"

import (
    "time"
    "unsafe"
    )

func loopStuff() {
    for {
        ptrs := make([]unsafe.Pointer, 20)
        for i:=0; i<20; i++ {
            ptrs[i] = C.malloc(16 * (1 << i))
        }
        time.Sleep(1 * time.Millisecond)
        for i:=0; i<20; i++ {
            C.free(ptrs[i])
        }
    }
}

func main() {
    for i:=0; i<10; i++ {
        go loopStuff()
    }
    loopStuff()
}

I'm on go 1.13.4 and ubuntu 18.04. What is going on here?

Answer 1

I think the problem lies with the for {} loop that is indefinitely running for each of the 10+1 goroutines.

Also if you read about free:

Deallocates the space previously allocated by malloc(), calloc(), aligned_alloc(), (since C11) or realloc().

So it basically deallocates the space and not necessarily returns memory back to the operating system immediately. Also, read this answer.

As the for loop is running indefinitely you'll definitely see a rise in memory which is not a leak though. I've modified your code by removing the for loop. And then I checked the RSS using BSD's top and it's using an averge of 1100KB and not rising. Maybe after waiting for sometime, it'll even reduce (when memory is returned back to the OS).

Tested with: OS: MacOS, Go: 1.15

BSD's top output:

PID    COMMAND      %CPU TIME     #TH  #WQ  #POR MEM   PURG CMPR PGRP  PPID  STATE
30767  cgo          0.0  00:00.00 11   0    20   1144K 0B   0B   30767 29146 sleeping

Code:

package main

// #include <stdlib.h>
import "C"

import "unsafe"

func loopStuff() {
        ptrs := make([]unsafe.Pointer, 20)
        for i := 0; i < 20; i++ {
                ptrs[i] = C.malloc(16 * (1 << i))
        }
        for i := 0; i < 20; i++ {
                C.free(ptrs[i])
        }
}

func main() {
        for i := 0; i < 10; i++ {
                go loopStuff()
        }
        loopStuff()
        // Block the main goroutine
        select {}
}

Update:

Try the following code. There was definitely allocation for a slice of unsafe.Pointer in every iteration. Also made a couple of changes. Perhaps on your system you won't see a memory leak now.

package main

/*
#include <stdlib.h>
*/
import "C"

import (
        "time"
        "unsafe"
)

func loopStuff() {
        ptrs := make([]*unsafe.Pointer, 20)
        for {
                for i := 0; i < 20; i++ {
                        tmp := C.malloc(16 * (1 << i))
                        ptrs[i] = &tmp
                }
                time.Sleep(1 * time.Millisecond)
                for i := 0; i < 20; i++ {
                        C.free(*ptrs[i])
                }
        }
}

func main() {
        for i := 0; i < 10; i++ {
                go loopStuff()
        }
        loopStuff()
}