ctypes - view c_char_p field of returned struct

Question

I've defined a simple C struct called TestStruct and a function init_struct to create an instance and return a pointer to it

#include <stdlib.h>
#include <stdio.h>

typedef struct {
    int x;
    int y;
    char* msg;
} TestStruct;

TestStruct* init_struct(int x, int y, char* msg) {
    TestStruct* p;
    TestStruct initial = {x, y, msg};
    p = malloc(sizeof(TestStruct));
    *p = initial;
    return p;
}

I compile the C code into a .so file using gcc. Then, in Python, I want to create a binding using ctypes that can access all the members of the C struct

import ctypes
import os

class PyStruct(ctypes.Structure):
    _fields_ = [('x', ctypes.c_int), 
                ('y', ctypes.c_int),         
                ('msg', ctypes.c_char_p)]

lib = ctypes.cdll.LoadLibrary(os.path.abspath('/path/to/libstruct.so'))
_init_struct = lib.init_struct
_init_struct.argtypes = [ctypes.c_int, ctypes.c_int, ctypes.c_char_p]
_init_struct.restype = ctypes.POINTER(PyStruct)

myStruct = _init_struct(1, 4, ctypes.c_char_p(b'hello world'))
print(myStruct.contents.x, myStruct.contents.y, myStruct.contents.msg)

The integer members of the struct (x and y) print out fine, but I can't figure out how to print the string that msg points to. Instead of the expected hello world, I end up seeing a bytes string b'\x01. My hunch from other reading is that I'm truncating the true, longer string and only showing the first byte.

Answer 1

You are passing ctypes.c_char_p(b'hello world') to init_struct and copying the pointer to the c_char_p block in the assignments to initial and p. However, that pointer to the c_char_p block is only valid for the duration of the call to init_struct, i.e., once init_struct returns, that c_char_p pointer will no longer be valid and accessing it will be undefined behavior. In other words, the copy of that pointer you took in myStruct.msg is dangling and should never be accessed outside init_struct.

Remember that ctypes does NOT violate Python's Garbage Collection (GC) rules. In this line myStruct = _init_struct(1, 4, ctypes.c_char_p(b'hello world')) ctypes will allocate some c_char_p object, copy in the string bhello world, null terminate it, and pass the raw pointer to that memory to the C side. Then the C side runs and your code takes a copy of that pointer. When the C side returns, ctypes releases its reference to the c_char_p object. Python's GC then finds that the c_char_p is no longer referenced and so it gets garbage collected. Hence, you end up with a dangling pointer in myStruct.msg.

The proper solution is to clone msg contents inside init_struct and provide a fini_struct function to free that clone memory when you are done with it, something like:

#include <stdlib.h>
#include <stdio.h>

typedef struct {
    int x;
    int y;
    char* msg;
} TestStruct;

TestStruct* init_struct(int x, int y, char* msg) {
    TestStruct* p = malloc(sizeof(TestStruct));
    p->x = x;
    p->y = y;
    p->msg = strdup(msg);
    return p;
}

void fini_struct(TestStruct* p) {
    free(p->msg);
    free(p);
}

Then the python side:

import ctypes
import os

class PyStruct(ctypes.Structure):
    _fields_ = [('x', ctypes.c_int), 
                ('y', ctypes.c_int),         
                ('msg', ctypes.c_char_p)]

lib = ctypes.cdll.LoadLibrary(os.path.abspath('/path/to/libstruct.so'))
_init_struct = lib.init_struct
_init_struct.argtypes = [ctypes.c_int, ctypes.c_int, ctypes.c_char_p]
_init_struct.restype = ctypes.POINTER(PyStruct)

_fini_struct = lib.fini_struct
_fini_struct.argtypes = [ctypes.POINTER(PyStruct)]

myStruct = _init_struct(1, 4, ctypes.c_char_p(b'hello world'))
print(myStruct.contents.x, myStruct.contents.y, myStruct.contents.msg)

# when you are done with myStruct
_fini_struct(myStruct)