Why does Ruby not add parent modules to the lexical scope for constant lookup when using double colons?

Question

This question is an extension of this question. The answer helped me understand what was happening, but I am still questioning why.

When defining two classes within a module there are two ways to write it.

Using Module Blocks:

module Foo
  class FirstClass
    def meth
      puts 'HELLO'
    end
  end

  class SecondClass
    def meth
      FirstClass.new.meth
    end
  end
end

Foo::SecondClass.new.meth

Using Double Colons:

module Foo; end

class Foo::FirstClass
  def meth
    puts 'HELLO'
  end
end

class Foo::SecondClass
  def meth
    FirstClass.new.meth
  end
end

Foo::SecondClass.new.meth

Both ways work for class definition, but when using double colons you cannot directly lookup FirstClass inside of SecondClass without including FirstClass or writing Foo::FirstClass. This happens because Foo is not a part of the lexical scope of SecondClass when it's defined with double colons, which can be demonstrated by using Module.nesting.

Why is Foo not added to the lexical scope with double colons? In the context of the lower level Ruby source code, why does ruby_cref point only to Foo::SecondClass instead of ruby_cref pointing to SecondClass which then points to Foo?

For Example:

+---------+       +---------+
| nd_next | <-----+ nd_next | <----+  ruby_cref
| nd_clss |       | nd_clss |
+----+----+       +----+----+
     |                 |
     |                 |
     v                 v
    Foo            SecondClass

Answer 1

Let me ask you the reverse question: why would it?

As you found in the last question, module nesting is important. For a quick intuitive example,

module Foo
  puts self       # executing in Foo context
  module Bar
    puts self     # executing in Foo::Bar context
  end
end

It is only modules (and its subclasses, such as Class) that can do this — change the execution context by nesting.

Now, over to your examples. The first snippet is effectively equivalent to:

module Foo
  # executing in Foo namespace context
  FirstClass = Class.new
  meth1 = proc do puts "HELLO" end
  FirstClass.define_method(:meth, meth1)
  SecondClass = Class.new
  meth2 = proc do FirstClass.new.meth end
  SecondClass.define_method(:meth, meth2)
  SecondClass.new.meth
end

Here, assuming we executed this at the main level, all of the references are relative to ::Foo. When we write FirstClass, it is understood as ::Foo::FirstClass.

The second snippet is effectively equivalent to

# executing in top namespace context
Foo = Module.new
Foo::FirstClass = Class.new
meth1 = proc do puts "HELLO" end
Foo::FirstClass.define_method(:meth, meth1)
Foo::SecondClass = Class.new
meth2 = proc do FirstClass.new.meth end        # ERROR
Foo::SecondClass.define_method(:meth, meth2)
Foo::SecondClass.new.meth

Written this way, it might be obvious why the second example does not work. If this was executed in main, then Foo::FirstClass that we defined is understood as ::Foo::FirstClass. The FirstClass mention in the error line is understood as ::FirstClass, which was never defined.

Answer 2

It allows you to explicitly set the modules that are used for constant lookup. Here's an example of a class MyClass defined under a module Foo:

module Foo
  A = 'A in Foo'
  B = 'B in Foo'
  C = 'C in Foo'
end

module Foo
  class MyClass
    B = 'B in MyClass'

    p Module.nesting  #=> [Foo::MyClass, Foo]

    def self.abc
      [A, B, C]
    end
  end
end

Foo::MyClass.abc
#=> ["A in Foo", "B in MyClass", "C in Foo"]

The constants are resolved the way Module.nesting shows, i.e. A, B, C are searched in Foo::MyClass and then in Foo.

---

Now for something more unusual. We can add a totally unrelated module Bar::Baz in-between Foo::MyClass and Foo for constant lookup:

module Foo
  A = 'A in Foo'
  B = 'B in Foo'
  C = 'C in Foo'
end

module Bar
  module Baz
    C = 'C in Bar::Baz'
  end
end

module Foo
  module ::Bar::Baz
    class Foo::MyClass
      B = 'B in MyClass'

      p Module.nesting  #=> [Foo::MyClass, Bar::Baz, Foo]

      def self.abc
        [A, B, C]
      end
    end
  end
end

Foo::MyClass.abc
#=> ["A in Foo", "B in MyClass", "C in Bar::Baz"]

I don't know if this has any real-world application, but it makes constant lookup extremely flexible. You can precisely select the modules you want to include (or exclude) for constant lookup and their order.