Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to terminate databricks cluster from notebook?

I want to terminate azure databricks cluster from notebook level. Is there any possibility?

like image 623
Splatt Avatar asked Jan 18 '26 07:01

Splatt


1 Answers

As suggested by Ganesh Chandrasekaran in comments. You can choose Job cluster for your requirement.

Job clusters terminate when your job ends, reducing resource usage and cost. The cluster is not terminated when idle but terminates only after all tasks using it have completed. If a shared job cluster fails or is terminated before all tasks have finished, a new cluster is created.

A cluster scoped to a single task is created and started when the task starts and terminates when the task is completed. In production, Databricks recommends using new shared or task scoped clusters so that each job or task runs in a fully isolated environment.

Refer - https://learn.microsoft.com/en-us/azure/databricks/data-engineering/jobs/jobs#choose-the-correct-cluster-type-for-your-job

like image 88
Abhishek Khandave Avatar answered Jan 20 '26 22:01

Abhishek Khandave



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!