Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python logging in multiprocessing

How to output logs to the console of several processes? Example:

import multiprocessing, logging, multiprocessing_logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
multiprocessing_logging.install_mp_handler(logger)

def worker():
    while True:
        logger.info("This is logging for TEST1")

def worker2():
    while True:
        logger.info("This is logging for TEST2")

if __name__ == '__main__':
    p1 = multiprocessing.Process(target=worker)
    p1.daemon = True
    p1.start()

    p2 = multiprocessing.Process(target=worker2)
    p2.daemon = True
    p2.start()

    p1.join()
    p2.join()

But output is incorrect:

INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST2
IINFO:root:This is logging for TEST1
NFO:root:This is logging for TEST2
INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST1
IINFO:root:This is logging for TEST2
NFO:root:This is logging for TEST1
IINFO:root:This is logging for TEST2
NFO:root:This is logging for TEST1

I tried to use multiprocessing-logging library, but it doesn't helped

like image 980
deniska369 Avatar asked Apr 15 '26 06:04

deniska369


1 Answers

Your (updated) code works fine over here:

INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST1
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2
INFO:root:This is logging for TEST2

If it doesn't for you then you should file a bug report with multiprocessing-logging.

like image 75
Florian Brucker Avatar answered Apr 16 '26 19:04

Florian Brucker