Boost Python 1.48 fails when exporting 32 bit enum

Question

Do newer versions of Boost Python support exporting full 32 bit enumerations from C++? I am using 1.48 and am able to export up to 30 bits but it fails every time due to "Access violation reading location 0x00000001" when I try and export a 32 bit number. Delving into the enum.cpp file, it appears that when object x is created, it sets byte 20 to 1 which causes the failure when it later tries to decrement p->name before it exists. Do I just need to upgrade to a newer version of boost python or is it just not possible to export a full 32 bit number? Thanks for your input!

Answer 1

EDIT: Found by Tigger at work, documenting here for posterity:

Problem is in the definition of the enum_object struct. The underlying digit storage in base_object allows for either 15-bit digit (as shorts) or 30-bit digit (as int) granularity, but the base_object only contains enough space for either 2x 15-bit digits or 1x 30-bit digits. When a > 30-bit enumeration value is stored the encoding requires either 3 shorts or 2 ints depending on the format compiled. The result is that the name member occupies the space needed for the extra digit storage. The reference decrement fails because the value is laid into place where the name member expects to be.

Solution is to add at least 32-bits of padding before the name member to handle 32-bit enumeration values. If 64-bit enumerations are required in a somewhat distant future then 2 pad words should be added.

struct enum_object
{
#if PY_VERSION_HEX >= 0x03000000
   PyLongObject base_object;
#else
   PyIntObject base_object;
#endif
   // ADD PADDING HERE TO FIX ALIGNMENT ISSUE
   PyObject* name;
};

END EDIT

TL/DR: The root error is either in the conversion or the assumed object alignment through a stack variable of the enum_base::add_value function, but it is probably just easiest to step through the converter to see if the value is mangled in some way. The other experiment I would try is to swap the order of the .value invocations (that might help determine if there is a stack-related issue).

Details: The enum_base::add_value function is unchanged between 1.48 and 1.58 (latest). Further, explicit conversion "(*this)(value)", based on python::api::object class and the object_base_initializer template also appears to be unchanged. I did not traverse further through the object_base constructor to see if anything changed there; I suggest stepping through the conversion sequence with the offending value to see if anything unusual occurs with the upper two-bits (I am not sure you will find anything there, but worth checking).

Remember that the converted value is then copied to the stack "object x = (this)(value);" and that the downcast for enum_object p is overlaying an object that is larger than the object referenced at x.ptr() (i.e., m_ptr member). I am not sure at this point if the m_ptr is stack or heap (didn't traverse those details), but either way if p is now overlaying stale or uninitialized memory then the reference decrement at p->name will fail if it does not point to valid memory (as you found).

I am not sure if the p->name reference decrement is necessary (the object lifetime is local)... I'll have to think on that some more. We can discuss more on Monday... it's a weekend... take a break (like I should talk!)

void enum_base::add_value(char const* name_, long value)
{
    // Convert name to Python string
    object name(name_);

    // Create a new enum instance by calling the class with a value
    object x = (*this)(value);

    // Store the object in the enum class
    (*this).attr(name_) = x;

    dict d = extract<dict>(this->attr("values"))();
    d[value] = x;

    // Set the name field in the new enum instanec
    enum_object* p = downcast<enum_object>(x.ptr());
    Py_XDECREF(p->name);
    p->name = incref(name.ptr());

    dict names_dict = extract<dict>(this->attr("names"))();
    names_dict[x.attr("name")] = x;
}