How to explain casting method reference to Iterable in for-each loop in Java?

Question

I found the following code:

try (Stream<String> lines = Files.lines(path, charset)) {
    for (String line : (Iterable<String>) lines::iterator) {
        //logic
    }
}

This code is compiled and works as expected. However, I can't understand this piece: (Iterable<String>) lines::iterator. As I understand a for-each loop is for (Foo foo: Iterable<Foo> iterable) {}. But how can method reference be casted to Iterable? Could anyone explain?

Answer 1

Java converts any method reference or lambda you see to an actual type: That type must be a so-called functional type, which is any interface where, if you take away any methods in java.lang.Object and also any methods with a default implementation, leaves precisely one method. Then that is a functional interface; you can mark it @FunctionalInterface but that's merely compiler-checked documentation: That act does not make it a functional interface; that act merely tells the compiler: Please generate a compiler error if I annotate a thing that is not a functional interface with it.

The thing is, Iterable is a functional interface: it is an interface that defines one method, with the signature: Given zero arguments, produce 1 Iterator.

And, turns out, lines::iterator also fits that signature: That is a method that takes no arguments, and returns one iterator.

Note that lines::iterator does not invoke the iterator() method of whatever lines is. Instead it encodes the idea of doing that. The value of lines::iterator is 'the idea of invoking the iterator() method on whatever lines is'. Whatever code is given 'this concept' can then choose to actually go ahead and do that. And then do it again. Or never do it. Or save it and do it tomorrow, in another thread.

That is, if it compiles at all - that is a method ref, so it must be used in a context where a functional type is required; you can't just write Object o = lines::iterator;.

Given the cast, there is now a context. So, that's fine then. (Iterable<String>) lines::iterator is valid java.

It even does the key semantic difference between Iterable and Iterator: Iterable is repeatable: You can 'start over' - just invoke the .iterator() method a second time which gets you a fresh new iterator that starts at the beginning. In contrast, an Iterator cannot go back. Once you invoke next() you have moved on. There is no prev() or reset() or goTo(0) or whatnot. You can invoke .iterator() on this Iterable<String> you have created more than once and the compiler won't stop you from writing that code.

Except, that's not actually useful: So this is a hack but a relatively innocuous one. Iterable instances are meant to be repeatable, but the docs of Stream's iterator() method is quite clear that it is not repeatable - once you walk through that iterator you can't do it again. The compiler won't complain if you invoke .iterator() more than once, but the runtime will - that second call will cause an exception. That's why Stream has an iterator() method, but does not implement Iterable<T>. Even though it trivially could - the compiler wouldn't stop Stream from doing that (as all instances of Iterable must have a method: public Iterator<T> iterator(), which.. Stream<T> does!. Not every requirement in an API can be captured by the type system, and 'must be invokable multiple times, returning a fresh, functioning iterator positioned at the start of whatever this is, every time you call it' is a thing Iterables should do, but which this does not, and which the type system cannot capture.

Hence, this produces a broken Iterable. So, letting it escape as part of a public api is a very very bad idea. However, in this snippet, it is used immediately so presumably the code can 'work around' the brokenness of it (the brokenness being: You can't invoke iterator() on it more than just the one time). The code really should have a comment explaining that this is a half-arsed iterable and therefore the code must not allow it to escape, nor can this code invoke iterator() on it more than once.