C# - XML Deserialization vs Binary Deserialization vs Binary+Deflate

Question

I'm saving a huge list of objects to a file, and later deserializing them. The resulting xml file can be about 3 gigs in size.

I want the Deserialization to be super fast, so i tried all three approaches (xml,binary,compressed)

Obviously, deserialization a compressed file takes much longer than an XML one. But i saw binary deserialization also taking a lot more time vs the xml deserialisation. Is that normal? Shoudn't both xml and binary take pretty much the same time to deserialize the object?

Also what do you think will be the best option to use in terms of a good balance between file size and deserialization speeds?

Answer 1

In this performance comparison between all sorts of serialization methods that come with .NET (BinaryFormatter, XmlSerializer, DataContractSerializer, etc.) and protobuf, the protobuf serializer seems to way ahead of the serializers that come with .NET. The resulting size appaears to be smaller as well. If the protobuf format is an option for you, I strongly recommend you have a look at it. :-)

Another option: if deserializing is slow, only deserialize the parts you really need. Create an index file that tells you the offsets of the objects you write to the data file, so you can quickly deserialize the objects you need in a random-access fashion.

Answer 2

Customise your serialisation, either fully or by implementing ISerializable and then using binary (though custom XML may also be worth experimenting with). Don't serialise memoised fields, but just the key field their value is based upon. Look for other areas where you can reduce size by serialising enough information to build part of the graph, rather than the full representation of the graph.

Then use deflate with that.