How to compare performance of messagepack-cli and json.net deserializers?

Question

I am trying to compare the performance of two different deserialisation methods in Unity3d, which is based on MonoDevelop's implementation of C# / .NET

Method A) Using MsgPack-CLI

Method B) Using NewtonSoft's Json.NET

Based on this blog post, I was under the impression that Messagepack would be faster for both reading and writing. However, I found that while the write performance is significantly better, the read performance is significantly slower.

For the purpose of testing deserialisation performance, am I doing something wrong in this code?

public void test_msgpack(int _num, Test_Class _obj)
{
    var serializer = MessagePackSerializer.Get< Test_Class >();
    var stream = new MemoryStream();

    serializer.Pack(stream, _obj);

    Stopwatch stopWatch = new Stopwatch ();
    stopWatch.Start ();
    while (_num > 0) {
        _num -= 1;
        stream.Position = 0;
        Test_Class deserializedObject = serializer.Unpack( stream );
    }

    stopWatch.Stop ();
    TimeSpan ts = stopWatch.Elapsed;
    string elapsedTime = String.Format ("{0:00}:{1:00}:{2:00}.{3:00}",
        ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10);

    print ("msgpack read: " + elapsedTime);
}

public void test_json(int _num, Test_Class _obj)
{
    var serializer = new JsonSerializer();
    var stream = new MemoryStream();

    var sw = new StreamWriter(stream);
    var JsonTextWriter = new JsonTextWriter(sw);
    var sr = new StreamReader(stream);
    var JsonTextReader = new JsonTextReader(sr);

    serializer.Serialize(JsonTextWriter, _obj);

    Stopwatch stopWatch = new Stopwatch ();
    stopWatch.Start ();
    while (_num > 0) {
        _num -= 1;
        stream.Position = 0;
        Test_Class deserializedObject = serializer.Deserialize<Test_Class>(JsonTextReader);
    }
    stopWatch.Stop ();
    TimeSpan ts = stopWatch.Elapsed;
    string elapsedTime = String.Format ("{0:00}:{1:00}:{2:00}.{3:00}",
        ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10);

    print ("json read: " + elapsedTime);
}

I am finding the JSON parsing performance to be roughly 100 times faster over 100000 iterations... which seems odd? Here is the form of the class that I am trying to serialise / deserialise:

public class Test_Class
{
    public string i { get; set; }
    public List<float> a { get; set; }
    public List<float> b { get; set; }
    public List<float> c { get; set; }
}

Answer 1

After some trial and error I found that the JSON.net code was returning a null object, thus the significanlty faster performance. I remedied this by modifying the test_json method as follows:

public void test_json(int _num, Test_Class _obj)
{
    JsonSerializer serializer = new JsonSerializer();

    MemoryStream stream = new MemoryStream();

    StreamWriter writer = new StreamWriter(stream);
    JsonTextWriter jsonWriter = new JsonTextWriter(writer);
    serializer.Serialize(jsonWriter, _obj);
    jsonWriter.Flush();

    Stopwatch stopWatch = new Stopwatch ();
    stopWatch.Start ();
    while (_num > 0) {
        _num -= 1;
        stream.Position = 0;

        StreamReader reader = new StreamReader(stream);
        JsonTextReader jsonReader = new JsonTextReader(reader);
        Test_Class deserialised_object = serializer.Deserialize<Test_Class>(jsonReader);
    }
    stopWatch.Stop ();
    TimeSpan ts = stopWatch.Elapsed;
    string elapsedTime = String.Format ("{0:00}:{1:00}:{2:00}.{3:00}",
        ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10);

    print ("json read: " + elapsedTime);
}

The jsonWriter.Flush() method is necessary, as is creating a new JsonTextReader on each read.

The results still show JSON.net as being faster with the deserialisation, however, this difference is now smaller. The MsgPack-CLI performance can apparently be improved by using a SerializerGenerator class, though I have yet to implement.