How to write type hints for an iterable abstract base class?

Question

I need to write an abstract base class for classes that:

• derive from an existing class, SomeClassIHaveToDeriveFrom (this is why I can't use a Protocol, I need this to be an abstract base class),
• implement the Iterable interface,
• contain objects of a specific type, Element (i.e. if we iterate over an instance, we get objects of type Element).

I tried to add a type hint to __iter__ in the abstract base class:

import abc
import collections.abc
import typing

class Element:
    pass

class SomeClassIHaveToDeriveFrom:
    pass

class BaseIterableClass(
    abc.ABC,
    collections.abc.Iterable,
    SomeClassIHaveToDeriveFrom,
):
    @abc.abstractmethod
    def __iter__(self) -> typing.Iterator[Element]:
        ...

class A(BaseIterableClass):
    def __iter__(self):
        return self

    def __next__(self):
        return "some string that isn't an Element"

a = A()
a_it = iter(a)
a_el = next(a)

But mypy doesn't detect any errors here, even though a is a BaseIterableClass instance that contains strs instead of Elements. I'm assuming that __iter__ is subject to name mangling, which means that the type hint is ignored.

How can I type hint BaseIterableClass so that deriving from it with an __iter__ function that iterates over something else than Element causes a typing error?

Answer 1

Running mypy in --strict mode actually tells you everything you need.

1) Incomplete Iterable

:13: error: Missing type parameters for generic type "Iterable"  [type-arg]

Since Iterable is generic and parameterized with one type variable, you should subclass it accordingly, i.e.

...
T = typing.TypeVar("T", bound="Element")
...
class BaseIterableClass(
    abc.ABC,
    collections.abc.Iterable[T],
    SomeClassIHaveToDeriveFrom,
):

2) Now we get a new error

:17: error: Return type "Iterator[Element]" of "__iter__" incompatible with return type "Iterator[T]" in supertype "Iterable"  [override]

Easily solvable:

...
    @abc.abstractmethod
    def __iter__(self) -> typing.Iterator[T]:

3) Now that we made BaseIterableClass properly generic...

:20: error: Missing type parameters for generic type "BaseIterableClass"  [type-arg]

Here we can specify Element:

class A(BaseIterableClass[Element]):
...

4) Missing return types

:21: error: Function is missing a type annotation  [no-untyped-def]
:24: error: Function is missing a return type annotation  [no-untyped-def]

Since we are defining the methods __iter__ and __next__ for A, we need to annotate them properly:

...
    def __iter__(self) -> collections.abc.Iterator[Element]:
...
    def __next__(self) -> Element:

5) Wrong return value

Now that we annotated the __next__ return type, mypy picks up that "some string that isn't an Element" is not, in fact, an instance of Element. 🙂

:25: error: Incompatible return value type (got "str", expected "Element")  [return-value]

Fully annotated code

from abc import ABC, abstractmethod
from collections.abc import Iterable, Iterator
from typing import TypeVar


T = TypeVar("T", bound="Element")


class Element:
    pass


class SomeClassIHaveToDeriveFrom:
    pass


class BaseIterableClass(
    ABC,
    Iterable[T],
    SomeClassIHaveToDeriveFrom,
):
    @abstractmethod
    def __iter__(self) -> Iterator[T]:
        ...


class A(BaseIterableClass[Element]):
    def __iter__(self) -> Iterator[Element]:
        return self

    def __next__(self) -> Element:
        return "some string that isn't an Element"  # error
        # return Element()

---

Fixed type argument

If you don't want BaseIterableClass to be generic, you can change steps 1)-3) and specify the type argument for all subclasses. Then you don't need to pass a type argument for A. The code would then look like so:

from abc import ABC, abstractmethod
from collections.abc import Iterable, Iterator


class Element:
    pass


class SomeClassIHaveToDeriveFrom:
    pass


class BaseIterableClass(
    ABC,
    Iterable[Element],
    SomeClassIHaveToDeriveFrom,
):
    @abstractmethod
    def __iter__(self) -> Iterator[Element]:
        ...


class A(BaseIterableClass):
    def __iter__(self) -> Iterator[Element]:
        return self

    def __next__(self) -> Element:
        return "some string that isn't an Element"  # error
        # return Element()

---

Maybe Iterator instead?

Finally, it seems that you actually want the Iterator interface, since you are defining the __next__ method on your subclass A. In that case, you don't need to define __iter__ at all. Iterator inherits from Iterable and automatically gets __iter__ mixed in, when you inherit from it and implement __next__. (see docs)

Also, since the Iterator base class is abstract already, you don't need to include __next__ as an abstract method.

Then the (generic version of the) code would look like this:

from abc import ABC
from collections.abc import Iterator
from typing import TypeVar


T = TypeVar("T", bound="Element")


class Element:
    pass


class SomeClassIHaveToDeriveFrom:
    pass


class BaseIteratorClass(
    ABC,
    Iterator[T],
    SomeClassIHaveToDeriveFrom,
):
    pass


class A(BaseIteratorClass[Element]):
    def __next__(self) -> Element:
        return "some string that isn't an Element"  # error
        # return Element()

Both iter(A()) and next(A()) work.

Hope this helps.