Huge List<double> save in small file

Question

I am trying to save a huge list of doubles in a file. For now it looks like this:

try{
       FileStream fs = new FileStream(saveFileDialog.FileName, FileMode.OpenOrCreate);                   
       using(BinaryWriter binaryWriter = new BinaryWriter(fs))
       {
             foreach (double value in bigData)
             {
                  binaryWriter.Write(value);
             }
             binaryWriter.Close();
        }
        fs.Close();
} catch(System.IO.FileNotFoundException)
{
    MessageBox.Show("Unexpected save error\n", "Save error!", MessageBoxButtons.OK);
}

bigData is a List<double> and in test case it contains 2 millions objects.

The saved file has around 15MB, which I think is quite a lot for only binary data. Has anyone got any idea, how I can make it much smaller?

Also, remember, that I need to open this file after saving - but this is done in another method.

Answer 1

A double is 8 bytes long, so 2 million times 8 is about 16MB. Seems OK.

Answer 2

The saved file has around 15MB, which I think is quite a lot for only binary data.

Well, a double is 8 bytes of data:

The Double value type represents a double-precision 64-bit number

You've got 2 million of them, so that means 16 million bytes. Seems about right to me.

Perhaps you actually want float values instead? That would save half the size... at the cost of precision and range, of course.

Compressing the data may help, but it may not - it depends on whether or not it contains a lot of repetitive information. You may find that compressing it increases the size rather than decreasing it - that's just the nature of having that many possible values.

Without knowing more about your context, we can't tell whether you really have 15MB of useful information or whether there's natural redundancy.