I'm trying to understand what is this variable called context in Airflow operators.
as example:
def execute(self, **context**).
Where it comes from? where can I set it? when and how can I use it inside my function? Another question is What is *context and **context? I saw few examples that uses this variable like this:
def execute(self, *context) / def execute(self, **context).
What is the difference and when should I use *context and **context
Documentation on the nature of context is pretty sparse at the moment. (There is a long discussion in the Github repo about "making the concept less nebulous".)
In a few places in the documentation it's referred to as a "context dictionary" or even an "execution context dictionary", but never really spelled out what that is.
Apparently, the Templates Reference is considered to be documentation for the context dictionary, although that's not actually mentioned on the page.
As is often the case with Airflow, a look at the source code is sometimes our best bet. One contributor has pointed to the following code block to describe the context dict:
return {
'conf': conf,
'dag': task.dag,
'dag_run': dag_run,
'ds': ds,
'ds_nodash': ds_nodash,
'execution_date': pendulum.instance(self.execution_date),
'inlets': task.inlets,
'macros': macros,
'next_ds': next_ds,
'next_ds_nodash': next_ds_nodash,
'next_execution_date': next_execution_date,
'outlets': task.outlets,
'params': params,
'prev_ds': prev_ds,
'prev_ds_nodash': prev_ds_nodash,
'prev_execution_date': prev_execution_date,
'prev_execution_date_success': lazy_object_proxy.Proxy(
lambda: self.get_previous_execution_date(state=State.SUCCESS)
),
'prev_start_date_success': lazy_object_proxy.Proxy(
lambda: self.get_previous_start_date(state=State.SUCCESS)
),
'run_id': run_id,
'task': task,
'task_instance': self,
'task_instance_key_str': ti_key_str,
'test_mode': self.test_mode,
'ti': self,
'tomorrow_ds': tomorrow_ds,
'tomorrow_ds_nodash': tomorrow_ds_nodash,
'ts': ts,
'ts_nodash': ts_nodash,
'ts_nodash_with_tz': ts_nodash_with_tz,
'var': {
'json': VariableJsonAccessor(),
'value': VariableAccessor(),
},
'yesterday_ds': yesterday_ds,
'yesterday_ds_nodash': yesterday_ds_nodash,
}
I've since found a mention of the context dictionary in the documentation! If anyone can find any more, I'll be happy to link them here.
When running your callable, Airflow will pass a set of keyword arguments that can be used in your function. This set of kwargs correspond exactly to the context variables you can use in your Jinja templates.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With