Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

snakemake - dynamically set log for each rule

I'd like to set the log file (log directive) for each of the workflow rules dynamically, so it contains the rule name. I tried to use the set_log() function like I do with set_params(), but am getting a cryptic error. Here is what I tried:

logs_dir = config['logs_dir']

rule1:
    ...

rule2:
    ...

for r in workflow.rules:
    r.set_log(os.path.join(logs_dir, r.name + '.log'))

and getting: AssertionError in file ... in line ...
I am probably misusing set_log, but couldn't figure it out. Any ideas?

UPDATE
Following the suggestions in the answers, I tried the following:

rule all:
    input:
        'b'

rule myrule:
    input:
        'a'
    output:
        'b'
    log:
        dummy = 'dummy.log'
    shell:
        """
        cp {input} {output}
        """

for r in workflow.rules:
    r.set_log(real=r.name + '.log')

Unfortunately, running it with snakemake -s snakefile -j 1 resulted again in an unspecified Assertion Error.

like image 424
soungalo Avatar asked Nov 03 '25 07:11

soungalo


1 Answers

The set_log function is defined here and I think it eventually runs into this assertion that throws this very-not-informative AssertionError. The way to fix it is to add an inline log definition to each rule:

logs_dir = config['logs_dir']

rule1:
    log: "rule1.log"
    ...

rule2:
    log: "rule2.log"
    ...

for r in workflow.rules:
    r.set_log(os.path.join(logs_dir, r.name + '.log'))

which unfortunately defeats the purpose of looping through and setting logs (if I'm understanding what you want to do here correctly). I recently ran into a very similar issue with setting threads without defining a threads value in each rule. Kind of annoying coming at it from this perspective but I think there is good reason behind this behavior.

One way you might be able to still avoid writing log: 'rule_name.log' in every rule would be using rule inheritance although I'm not totally sure how passing the names and wildcards would work. If you get something like this working please let me know.

like image 153
Ulthran Avatar answered Nov 05 '25 07:11

Ulthran



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!