How to serialize big objects in .NET? (OutOfMemory Exceptions)

Question

I'm using serialization for "save" feature in my application. But when the data is too big (15+ MB) I'm starting to get OutOfMemory exceptions.

I've got so many objects and they are connected with other little objects, I think this is causing too much processing power and data held in the memory.

My code is based on this, almost same:

http://www.codeproject.com/KB/vb/TreeViewDataAccess.aspx

Edit :

1. I don't use custom serialization, it's all done by [Serialization] attributes. Excluding some fields.

2. I serialize so many objects and custom classes. Includes Dictionary, structures and bunch of other stuff.

3. I serialize it into a file.

4. I use XmlSerializer

P.S. I've got 4 GB physical memory.

Solution

Thanks to answers, my problem was found to be with XmlSerializer and I've got rid of it. Binary serialization is working just fine with the data I've got.

Answer 1

I had exactly the same problem. The reason is that .NET serialization does not scale.

I solved the problem by using Simon Hewitt's excellent open source library, see Optimizing Serialization in .NET - part 2.

Besides dramatically reducing memory usage it is also much faster. Similar to the article I got a speed-up of 20 times.

Answer 2

Actually, XmlSerializer ignores the SerializableAttribute attributes. They're used only by the formatting classes (BinaryFormatter, SoapFormatter).

I wouldn't serialize using the XmlSerializer, and especially not a combination of XmlSerializer and BinaryFormatter.

I would simply try to serialize everything using the BinaryFormatter.