How to reuse a cluster with Databricks Asset bundles

Question

I am using Databricks asset bundles as an IAC tool with databricks. I want to create a cluster using DAB and then reuse the same cluster in multiple jobs. I can not find an example for this. Whatever examples I found out have all specified individual new clusters while defining a job. How can we reuse the clusters?

Answer 1

The only way to "reuse" a cluster is when you create a general purpose cluster ahead of time and specify its id in your asset-bundle task.

    - task_key: my_task_id
      existing_cluster_id: ${var.existing_cluster_id}

The job clusters cannot be re-used across jobs, instead a new one is created whenever you deploy your asset bundle.