Apply python logging format to each line of a message containing new lines

Question

I would like to include an id associated with the request for every log line in a log file. I have the mechanism for retrieving the request id via a logging filter working fine.

My problem is that when a log line contains a newline, the line of course get wrapped onto a "bare" line. Is there a way to tell the logging library to split the message, and apply the format to each element of the split?

import logging

logger = logging.getLogger()
handler = logging.StreamHandler()
formatter = logging.Formatter(
        "%(requestid)s\t%(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)

logger.debug("blah\nblah")

Output:

xxx blah
blah

Desired output:

xxx blah
xxx blah

Answer 1

Though this is not the best way to go about this, but this will help you without changing a lot of code.

import logging


class CustomHandler(logging.StreamHandler):
    def __init__(self):
        super(CustomHandler, self).__init__()

    def emit(self, record):
        messages = record.msg.split('\n')
        for message in messages:
            record.msg = message
            super(CustomHandler, self).emit(record)


log = logging.getLogger()
handler = CustomHandler()
formattor = logging.Formatter("xxx:%(message)s")

handler.setFormatter(formattor)

log.addHandler(handler)
log.setLevel(logging.DEBUG)

if __name__ == '__main__':
    log.debug('hello\nhi')

Output:

xxx:hello
xxx:hi