Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to write to delta table/delta format in Python without using Pyspark?

I am looking for a way to write back to a delta table in python without using pyspark. I know there is a library called deltalake/delta-lake-reader that can be used to read delta tables and convert them to pandas dataframes.

The goal is to write back to the opened delta table

The input code looks like this:

from deltalake import DeltaTable
dt = DeltaTable('path/file')
df = dt.to_pandas()

So is there any way to get something like this to write from a pandas dataframe back to a delta table:

df = pandadf.to_delta()
DeltaTable.write(df, 'path/file')

Thank you for your assistance!

like image 647
FRITTENPIET Avatar asked Jan 17 '26 21:01

FRITTENPIET


1 Answers

Now it is supported !!!, see this example

import duckdb 
from deltalake.writer import write_deltalake
df =duckdb.sql('''
LOAD 'httpfs';
SELECT countries_and_territories, sum(deaths) as total FROM 
read_parquet('https://pandemicdatalake.blob.core.windows.net/public/curated/covid-19/ecdc_cases/latest/ecdc_cases.parquet')
group by 1
order by total desc
limit 5;
''').df()
write_deltalake('Pathto/covid', df,mode='append')
like image 153
Mim Avatar answered Jan 19 '26 18:01

Mim



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!