Evaluate statements from within Python logging YAML config file

Question

Consider the following snippet of a Python logging YAML config file:

version: 1
formatters:
  simple:
    format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
handlers:
  logfile:
    class: logging.handlers.TimedRotatingFileHandler
    level: DEBUG
    filename: some_fancy_import_name.generate_filename_called_error
    backupCount: 5
    formatter: simple

I would like to load this YAML config file this way:

with open('logging.yaml', 'r') as fd:
    config = yaml.safe_load(fd.read())
logging.config.dictConfig(config)

Take special notice of the filename to which the handler should write logs. In normal Python code, I would expect some_fancy_import_name.generate_filename_called_errorlog to generate the string 'error.log'. All in all, I would like to say that this logging handler should write to the file 'error.log' in the current directory.

However, as it turns out, this is not the case. When I look at the current directory, I see a file named 'some_fancy_import_name.generate_filename_called_errorlog'.

Why go through all this trouble?

I would like filename to be programmatically determined. I have successfully tried configuring logging using normal Python scripting this way:

# fancy import name
from os import environ as env

# Programmatically determine filename path
log_location = env.get('OPENSHIFT_LOG_DIR', '.')
log_filename = os.path.join(log_location, 'error')
handler = logging.handlers.TimedRotatingFileHandler(log_filename)

See how the log_filename path was inferred from environment variables.

I would like to translate this to a YAML config file. Is it possible?

Perhaps I might need to dig through the dict produced by yaml.safe_load(fd.read()) and do some eval() stuff?

Answer 1

You can add a custom constructor and mark the value with a special tag, so your constructor gets executed when loading it:

import yaml

def eval_constructor(loader, node):
  return eval(loader.construct_scalar(node))

yaml.add_constructor(u'!eval', eval_constructor)

some_value = '123'

config = yaml.load("""
version: 1
formatters:
  simple:
    format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
handlers:
  logfile:
    class: logging.handlers.TimedRotatingFileHandler
    level: DEBUG
    filename: !eval some_value
    backupCount: 5
    formatter: simple
""")

print config['handlers']['logfile']['filename']

This prints 123, since the value some_value has the tag !eval, and therefore is loaded with eval_constructor.

Be aware of the security implications of evaling configuration data. Arbitrary Python code can be executed by writing it into the YAML file!