Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Databricks: how to convert Spark dataframe under %python to dataframe under %r

I have a Dataframe in %python environment, and try to use that in %r environment.

How can I convert a spark dataframe under %python to %r ?

like image 640
Hello lad Avatar asked Oct 31 '25 16:10

Hello lad


1 Answers

It is not even worth trying. The idiomatic way is to register temporary view and let metastore do the rest. In Python:

df.createOrReplaceTempView("some_name")

in R:

sql("REFRESH TABLE some_name")
df <- sql("SELECT * FROM some_name")
like image 105
zero323 Avatar answered Nov 04 '25 01:11

zero323



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!