How does Scala's type erasure work for higher kinded type parameters?

Question

I don't understand which generic type parameters Scala erases. I used to think that it should erase all generic type parameters, but this does not seem to be the case.

Correct me if I'm wrong: if I instantiate an instance of type Map[Int, String] in the code, then at the runtime, the instance knows only that it is of type Map[_, _], and does not know anything about its generic type parameters. This is why the following succeeds to compile and to execute without errors:

val x: Map[Int, String] = Map(2 -> "a")
val y: Map[String, Int] = x.asInstanceOf[Map[String, Int]]

Now I would expect that all higher-kinded type parameters are also erased, that is, if I have a

class Foo[H[_, _], X, Y]

I would expect that an instance of type Foo[Map, Int, String] knows nothing about Map. But now consider the following sequence of type-cast "experiments":

import scala.language.higherKinds

def cast1[A](a: Any): A = a.asInstanceOf[A]
def cast2[H[_, _], A, B](a: Any) = a.asInstanceOf[H[A, B]]
def cast3[F[_[_, _], _, _], H[_, _], X, Y](a: Any) = a.asInstanceOf[F[H, X, Y]]
class CastTo[H[_, _], A, B] {
  def cast(x: Any): H[A, B] = x.asInstanceOf[H[A, B]]
}

ignoreException {
  val v1 = cast1[String](List[Int](1, 2, 3))                      
  // throws ClassCastException
}

ignoreException {
  val v2 = cast2[Map, Int, Long](Map[String, Double]("a" -> 1.0)) 
  // doesn't complain at all, `A` and `B` erased
}

ignoreException {
  // right kind
  val v3 = cast2[Map, Int, Long]((x: Int) => x.toLong)            
  // throws ClassCastException
}

ignoreException {
  // wrong kind
  val v4 = cast2[Map, Int, Long]("wrong kind")                     
  // throws ClassCastException
}

ignoreException {
  class Foo[H[_, _], X, Y](h: H[X, Y])
  val x = new Foo[Function, Int, String](n => "#" * n)
  val v5 = cast3[Foo, Map, Int, Long](x)
  // nothing happens, happily replaces `Function` by `Map`
}

ignoreException {
  val v6 = (new CastTo[Map, Int, Long]).cast(List("hello?"))       
  // throws ClassCastException
}

ignoreException {
  val castToMap = new CastTo[Map, Int, Long]
  val v7 = castToMap.cast("how can it detect this?")               
  // throws ClassCastException
}

ignoreException {
  val castToMap = new CastTo[Map, Int, Long]
  val madCast = castToMap.asInstanceOf[CastTo[Function, Float, Double]]
  val v8 = madCast.cast("what does it detect at all?")
  // String cannot be cast to Function???
  // Why does it retain any information about `Function` here?
}


// --------------------------------------------------------------------
var ignoreBlockCounter = 0
/** Executes piece of code, 
  * catches an exeption (if one is thrown),
  * prints number of `ignoreException`-wrapped block,
  * prints name of the exception.
  */
def ignoreException[U](f: => U): Unit = {
  ignoreBlockCounter += 1
  try {
    f
  } catch {
    case e: Exception =>
      println("[" + ignoreBlockCounter + "]" + e)
  }
}

Here is the output (scala -version 2.12.4):

[1]java.lang.ClassCastException: scala.collection.immutable.$colon$colon cannot be cast to java.lang.String
[3]java.lang.ClassCastException: Main$$anon$1$$Lambda$143/1744347043 cannot be cast to scala.collection.immutable.Map
[4]java.lang.ClassCastException: java.lang.String cannot be cast to scala.collection.immutable.Map
[6]java.lang.ClassCastException: scala.collection.immutable.$colon$colon cannot be cast to scala.collection.immutable.Map
[7]java.lang.ClassCastException: java.lang.String cannot be cast to scala.collection.immutable.Map
[8]java.lang.ClassCastException: java.lang.String cannot be cast to scala.Function1

• The cases 1, 3, 4 indicate that asInstanceOf[Foo[...]] does care about Foo, this is expected.
• The case 2 indicates that asInstanceOf[Foo[X,Y]] does not care about X and Y, this is also expected.
• The case 5 indicates that asInstanceOf does not care about higher kinded type parameter Map, similar to case 2, this is also expected.

So far so good. However, the cases 6, 7, 8 suggest a different behavior: here, an instance of type CastTo[Foo, X, Y] seems to retain information about the generic type parameter Foo for some reason. More precisely, a CastTo[Map, Int, Long] seems to carry around enough information with it to know that a string cannot be cast into a Map. Moreover, in case 8, it seems to even change Map to Function because of a cast.

Question(s):

1. Is my understanding correct that the first generic parameter of CastTo is not erased, or is there something else what I don't see? Some implicit operation or anything?
2. Is there any documentation that describes this behavior?
3. Is there any reason why I should want this behavior? I find it somewhat counter-intuitive, but maybe it's just me, and I'm using the tool wrong...

Thanks for reading.

EDIT: Poking around in similar examples revealed an issue with the 2.12.4-compiler (see my own "answer" below), but this is a separate issue.

Answer 1

I think you are confusing some things.

Casts to generic types are deferred until the point where the types become concrete. For example, take this piece of code:

class CastTo[H[_, _], A, B] {
  def cast(x: Any): H[A, B] = x.asInstanceOf[H[A, B]]
}

In the bytecode you can only cast to a real class, because it doesn't know anything about generics. So the above will, in bytecode, be roughly equivalent to:

class CastTo {
  def cast(x: Object): Object = x
}

Then later in the code you give a String to method cast and the compiler can see that according to the type information it has, a Map[Int, Long] will come out. But in the bytecode cast has an erased return type of Object, so the compiler has to insert a cast at the use-site of the cast method. This code

val castToMap = new CastTo[Map, Int, Long]
val v7 = castToMap.cast("how can it detect this?")

will, in the bytecode, be roughly equivalent to the following (pseudo) code:

val castToMap = new CastTo
val v7 = castToMap.cast("how can it detect this?").asInstanceOf[Map]

As for your other questions:

2. Not that I immediately know of.
3. Why would you not want it? You are casting a String to a Map[Int, Long]. That is bound to crash eventually. Failing (relatively) fast with a ClassCastException is probably the safest, most user-friendly option.