How to start Airflow Dag with a past Data Interval Date

Question

I am working in Ariflow 2.2.3 and I can't figure out how to trigger my dag with a past execution date. When I click Trigger dag with Config, I changed the calendar to the date I wanted but when I clicked run, I saw the run but it didn't run.

I also tried putting the date in the config section with {"start_date":"date"} but that didn't work aswell.

Any idea how to trigger a dag with a date in the past?

Answer 1

To create a past Airflow run you have multiple option, but most of them needs to update the start date of your dag to be older than the date of the desired run date (first four options), otherwise the run will be marked as succeeded without being executed.

1. Via Airflow UI: you can click on the run icon, and choose trigger DAG w/ config, then choose the logical date you want, and create the run.
2. Via Airflow REST API: here is the doc
3. Via Airflow python lib:

   DagBag(read_dags_from_db=True).get_dag(<your dag name>).create_dagrun(
       run_id=<run id>,
       run_type=DagRunType.MANUAL,
       execution_date=<execution date>,
       state=State.QUEUED,
       external_trigger=True,
       data_interval=(<desired start date>, <desired end date>),
       conf={<any conf>: <any value>, ...},
   )
   

4. Via Airflow CLI: here is the doc

   airflow dags trigger -e <execution date> <dag id>
   

5. Via Airlfow CLI backfill command: here is the doc. It doesn't need an old start date, where the run and the task are managed by the CLI and not the Airflow scheduler.

   airflow dags backfill -s <start date> -e <end date> <dag id>
   

   P.S: this concept is based on schedule interval, so if you want a signle run, should be + , otherwise you will have multiple runs between the two days. This is different in the option 3 which create a single run with the and you provided.