R error allocMatrix

Question

HI all,

I was trying to load a certain amount of Affymetrix CEL files, with the standard BioConductor command (R 2.8.1 on 64 bit linux, 72 GB of RAM)

abatch<-ReadAffy()

But I keep getting this message:

Error in read.affybatch(filenames = l$filenames, phenoData = l$phenoData,  : 
  allocMatrix: too many elements specified

What's the general meaning of this allocMatrix error? Is there some way to increase its maximum size?

Thank you

Answer 1

The problem is that all the core functions use INTs instead of LONGs for generating R objects. For example, your error message comes from array.c in /src/main

if ((double)nr * (double)nc > INT_MAX)
    error(_("too many elements specified"));

where nr and nc are integers generated before, standing for the number of rows and columns of your matrix:

nr = asInteger(snr);
nc = asInteger(snc);

So, to cut it short, everything in the source code should be changed to LONG, possibly not only in array.c but in most core functions, and that would require some rewriting. Sorry for not being more helpful, but i guess this is the only solution. Alternatively, you may wait for R 3.x next year, and hopefully they will implement this...

Answer 2

If you're trying to work on huge affymetrix datasets, you might have better luck using packages from aroma.affymetrix.

Also, bioconductor is a (particularly) fast moving project and you'll typically be asked to upgrade to the latest version of R in order to get any continued "support" (help on the BioC mailing list). I see that Thrawn also mentions having a similar problem with R 2.10, but you still might think about upgrading anyway.