Creating luigi parameters from other parameters at initialization

Question

I have the following question - can I use the value of one parameter to define another parameter ? Here's an illustration of what I'm trying to do. Suppose I have a config file that looks like this:

[MyTaskRunner]
logdir=/tmp/logs
numruns=2

and I defined MyTaskRunner like this:

class MyTaskRunner(luigi.Task):
      logdir=luigi.Parameter(default=None)
      rundate=luigi.Parameter(default=today)

where logdir is a parameter obtained from the config file and rundate is a parameter that was passed in at runtime.

Now, suppose I wish to define a new variable logpath_str like this

       logpath_str="{}/{}".format(logdir, rundate)

Is it possible to define this as a parameter ?

Would the solution be to specify the default value as in:

       logpath=luigi.Parameter(default=logpath_str)

Any suggestions welcome.

Answer 1

Parameter values are not resolved until the class has been initialised (during __init__) so a simple way to achieve the behaviour you're looking for is to implement __init__ yourslef and initialise logpath with a default value after calling super.

class MyTaskRunner(luigi.Task):
    logdir=luigi.Parameter(default=None)
    logpath=luigi.Parameter(default=None)
    rundate=luigi.Parameter(default=today)

    def __init__(self, *args, **kwargs):
        super(MyTaskRunner, self).__init__(*args, **kwargs)

        if self.logpath is None:
            self.logpath = "{}/{}".format(self.logdir, self.rundate)