_UnixSelectorEventLoop has no attribute _closed (nor other instance attributes) while being deleted in Jupyter Notebook

Question

There are a few questions similiar to this but not a duplicate I could find.

I'm doing some automatic unit testing of Python code and running these tests in a Jupyter Notebook. On certain computers (Macs in our computer labs) but not on others (my own Windows machine or some other personal Macs) students see a warning about an ignored error along with their test output. The error boils down to: a _UnixSelectorEventLoop object is being deleted, and in its __del__ method, there's an if self.is_closed(): test, where is_closed is a method that checks the self._closed instance variable. For some reason, the _UnixSelectorEventLoop object in question does not even have a _closed attribute, causing an AttributeError which is ignored and printed as a warning due to being raised in the context of deletion.

When I add prints to the __del__ method I can see this is happening on a return line, but enabling gc debugging seems to indicate it isn't the result of cyclic garbage collection. My next thought is that a local variable is going out of scope, but I haven't found one via print(locals()) which is a _UnixSelectorEventLoop although maybe I'm not looking hard enough.

In any case, printing dir(self) during the __del__ method shows that NONE of the expected instance variables exist, almost as if this is an instance which never got initialized. When I print the ID of self during __del__ and add a similar print to __init__, I see the same number of __init__ and __del__ calls (perhaps just a coincidence) but the IDs on __init__ don't match up. My guess is the object gets copied somewhere in an incomplete way. The __init__ calls I do see are coming from tornado, but I haven't dealt with that directly before.

Have others come across and solved similar issues with __del__ on an object getting a hollowed-out object missing instance variables that should be set by __init__, particularly in async contexts? Any hints would be appreciated!

Python version is 3.10.

I don't have an MWE right now, although I'll try to find time to cook one up. The fact that it's machine-dependent means I'm not confident in reproducing it, and it only happens in Jupyter Notebook contexts, not running code in normal Python.

I can try to narrow down a bit the parts of my code that trigger it, but I'm worried that the actual trigger is happening via a thread handoff and has nothing to do with the traceback I can print out from within __del__.

Answer 1

After wasting 3 days on this issue I can offer a terrible but efficient hack: to monkey-patch the __del__ method of the BaseEventLoop class. This was inspired by Andrew Clark's comment to this SO answer:

import asyncio

# shorthand for the class whose __del__ raises the exception
_BEL = asyncio.base_events.BaseEventLoop

_original_del = _BEL.__del__

def _patched_del(self):
    try:
        # invoke the original method...
        _original_del(self)
    except:
        # ... but ignore any exceptions it might raise
        # NOTE: horrible anti-pattern
        pass

# replace the original __del__ method
_BEL.__del__ = _patched_del

I would love to file a bug report but I don't know who to contact: the IPython devs, the IPyKernel devs or the Jupyter notebook devs. My use case is somewhat convoluted, currently cannot provide a simple example that triggers the bug.