read() from a ExFileObject always cause StreamError exception

Question

I am trying to read only one file from a tar.gz file. All operations over tarfile object works fine, but when I read from concrete member, always StreamError is raised, check this code:

import tarfile
fd = tarfile.open('file.tar.gz', 'r|gz')
for member in fd.getmembers():
    if not member.isfile():
        continue
    cfile = fd.extractfile(member)
    print cfile.read()
    cfile.close()
fd.close()

cfile.read() always causes "tarfile.StreamError: seeking backwards is not allowed"

I need to read contents to mem, not dumping to file (extractall works fine)

Thank you!

Answer 1

The problem is this line:

fd = tarfile.open('file.tar.gz', 'r|gz')

You don't want 'r|gz', you want 'r:gz'.

If I run your code on a trivial tarball, I can even print out the member and see test/foo, and then I get the same error on read that you get.

If I fix it to use 'r:gz', it works.

From the docs:

mode has to be a string of the form 'filemode[:compression]'

...

For special purposes, there is a second format for mode: 'filemode|[compression]'. tarfile.open() will return a TarFile object that processes its data as a stream of blocks. No random seeking will be done on the file… Use this variant in combination with e.g. sys.stdin, a socket file object or a tape device. However, such a TarFile object is limited in that it does not allow to be accessed randomly, see Examples.

'r|gz' is meant for when you have a non-seekable stream, and it only provides a subset of the operations. Unfortunately, it doesn't seem to document exactly which operations are allowed—and the link to Examples doesn't help, because none of the examples use this feature. So, you have to either read the source, or figure it out through trial and error.

But, since you have a normal, seekable file, you don't have to worry about that; just use 'r:gz'.