Python3 compress logger module logs on the fly

Question

I can't find a way to compress the logs I write with logger module.

For example:

import logging
import gzip

logger = logging.getLogger('')
z_file = gzip.open('out.log.gz', mode='wb')
logger.addHandler(logging.StreamHandler(z_file))
logger.warning("test".encode("UTF-8"))

Both codecs.open and gzip.open give me

--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3.4/logging/__init__.py", line 966, in emit
    stream.write(msg)
  File "/usr/lib/python3.4/gzip.py", line 343, in write
    self.crc = zlib.crc32(data, self.crc) & 0xffffffff
TypeError: 'str' does not support the buffer interface

When I'm trying to use their handlers. What am I doing wrong?

Related question that doesn't include logger module: Writing append only gzipped log files in Python

Answer 1

Specify encoding to gzip.open. You should use explicit text mode (wt) to specify encoding. And just pass a string to the logging methods.

import logging
import gzip

logger = logging.getLogger('')
z_file = gzip.open('out.log.gz', mode='wt', encoding='utf-8')
logger.addHandler(logging.StreamHandler(z_file))
logger.warning("test")