PyBind11 destructor not invoked?

Question

I have a c++ class wrapped with PyBind11. The issue is: when the Python script ends the c++ destructor is not being automatically invoked. This causes an untidy exit because networking resources need to be released by the destructor.

As a work-around it is necessary to explicitly delete the Python object, but I don't understand why!

Please could someone explain what is wrong here and how to get the destructor called automatically when the Python object is garbage collected?

Pybind11 binding code:

py::class_<pcs::Listener>(m, "listener")
    .def(py::init<const py::object &, const std::string &, const std::string &, const std::string &, const std::string &, const std::set<std::string> &, const std::string & , const bool & , const bool & >(), R"pbdoc(
    Monitors network traffic.

    When a desired data source is detected a client instance is connected to consume the data stream.

    Reconstructs data on receipt, like a jigsaw.  Makes requests to fill any gaps.  Verifies the data as sequential.

    Data is output by callback to Python.  Using the method specified in the constructor, which must accept a string argument.
)pbdoc");

In Python:

#Function to callback
def print_string(str):
    print("Python; " + str)

lstnr = listener(print_string, 'tcp://127.0.0.1:9001', clientCertPath, serverCertPath, proxyCertPath, desiredSources, 'time_series_data', enableCurve, enableVerbose)

#Run for a minute
cnt = 0
while cnt < 60:
    cnt += 1
    time.sleep(1)

#Need to call the destructor explicity for some reason    
del lstnr

Answer 1

So some years later I fixed this issue by enabling Python context manager with support by adding __enter__ and __exit__ method handling to my PyBind11 code:

py::class_<pcs::Listener>(m, "listener")
.def(py::init<const py::object &, const std::string &, const std::string &, const std::string &, const std::string &, const std::set<std::string> &, const std::string & , const bool & , const bool & >(), R"pbdoc(
    Monitors network traffic.

    When a desired data source is detected a client instance is connected to consume the data stream.
    
    Specify 'type' as 'string' or 'market_data' to facilitate appropriate handling of BarData or string messages.

    Reconstructs data on receipt, like a jigsaw.  Makes requests to fill any gaps.  Verifies the data as sequential.

    Data is output by callback to Python.  Using the method specified in the constructor, which must accept a string argument.
)pbdoc")
.def("__enter__", &pcs::Listener::enter, R"pbdoc(
    Python 'with' context manager support.
)pbdoc")    
.def("__exit__", &pcs::Listener::exit, R"pbdoc(
    Python 'with' context manager support.
)pbdoc");

Added corresponding functions to the C++ class, like so:

//For Python 'with' context manager
auto enter(){std::cout << "Context Manager: Enter" << std::endl; return py::cast(this); }//returns a pointer to this object for 'with'....'as' python functionality
auto exit(py::handle type, py::handle value, py::handle traceback){ std::cout << "Context Manager: Exit: " << type << " " << value << " " << traceback <<  std::endl; }

N.B.

1. The returned pointer value from enter() is important to the as functionality in a with....as statement.

2. The parameters passed to exit(py::handle type, py::handle value, py::handle traceback) are useful debugging info.

Python usage:

with listener(cb, endpoint, clientCertPath, serverCertPath, proxyCertPath, desiredSources, type, enableCurve, enableVerbose):
    cnt = 0
    while cnt < 10:
        cnt += 1
        time.sleep(1)

The Python context manager now calls the pcs::Listener::exit on the C++ object, which should release the networking resources.