How to use a single Python logging object throughout the Python application?

Question

Python provides the logging module. We can use the logger in place of print and use its multiple log levels. The issue here is that when we use logger, we pass the log string into the logger object. This means that the logger object must be accessible from every function/method and class in the entire Python program.

logger = logging.getLogger('mylogger')
logger.info('This is a message from mylogger.')

Now my question is, for large Python programs that are possibly split across more than 1 source file and made up of a multitude of functions/methods and classes, how do we ensure that the same logger object is used everywhere to log messages? Or do I have the wrong idea on how the logging module is used?

Answer 1

Or do I have the wrong idea on how the logging module is used?

Yup, wrong idea.

for large Python programs that are ... split across more than 1 source file ... how do we ensure that the same logger object is used everywhere ?

That's not a goal.

We log messages so a maintenance engineer can pick up the pieces later.

Use one logger per module.

It is a feature that a logger will reveal the source module it came from. We use that to rapidly narrow down "what code ran?" & "with what input values?" This assists a maintainer in reproducing and repairing observed buggy output.

---

The usual idiom is to begin each module with

logger = logging.getLogger(__name__)

That way, logger.error(...) and similar calls will reveal the name of the module reporting the error, which lets maintainers rapidly focus on the code leading up to the error.

---

We do desire that all loggers follow the same output format, so that all loggers produce compatible messages that can be uniformly parsed by e.g. some awk script.

Typically an "if main" clause invokes basicConfig, just once, which takes care of that.

Answer 2

I can tell you what I do! I create a module in my application called logs that is imported by the entrypoint, creates and configures a root logger, and exposes a get_logger function that gets a child of that root logger with a given name. Consider:

# logs.py
import logging

ROOT = logging.getLogger("root")
# configure that logger in some way

def get_logger(name: str):
    return ROOT.getLogger(name)

Then in other modules, I do:

# lib.py
import logs

logger = logs.get_logger(__name__)

# the body of my module, calling logger.info or etc as needed

The important part to remember is that logs.py cannot have dependencies on anything else in the package or else you have to be very careful the order in which you import things, lest you get a circular dependency problem!