Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kryo serialization refuses to register class

Tags:

scala

kryo

I'm trying to use kryo serialization with:

kryo.setRegistrationRequired(true);

I keep getting the following error saying that a certain class is not registered:

java.lang.IllegalArgumentException: Class is not registered: com.my.package.MyClass[]
Note: To register this class use: kryo.register(com.my.package.MyClass[].class);

However, I do register it:

kryo.register(classOf[MyClass[_]])

When I set Log.TRACE() I get the following output:

00:11 TRACE: [kryo] Register class ID 51: com.my.package.MyClass (com.esotericsoftware.kryo.serializers.FieldSerializer)

Why does it say it is not registered, when the trace logger prints that it has been registered. I cannot find any useful documentation on the matter. Has anyone experienced this before? If it helps i'm running Apache Spark v0.8.1

like image 789
kbirk Avatar asked Feb 25 '14 22:02

kbirk


1 Answers

I figured out what the issue was. I was mistaking the java array syntax [] for the scala generic syntax []. The exception was being being thrown since I did not register an array of MyClass.

So in java

kryo.register( MyClass[].class );

And scala

kryo.register( classOf[ Array[ MyClass[_] ] ] )
like image 93
kbirk Avatar answered Sep 21 '22 19:09

kbirk