I am trying to use the round() function in databricks to round some float values to 2 digits. However, the databricks python is not working like normal python.
Please help me with the reasons and solutions if any.
lis = [-12.1334, 12.23433, 1.2343, -104.444]
lis2 = [round(val,2) for val in lis]
print(lis2)
TypeError: Invalid argument, not a string or column: -12.1334 of type <type 'float'>. For column literals, use 'lit', 'array', 'struct' or 'create_map' function.
Image Proof of Code
This is only reproducible when you import the spark round function from the function module in spark.sql
The spark round function requires a string or a column. Which explains the error.
You can either alias the import such as import pyspark.sql.functions as F instead of from pyspark.sql.functions import *
You can get the origin round method this way.
import builtins
round = getattr(builtins, "round")
And then you can execute
lis = [-12.1334, 12.23433, 1.2343, -104.444]
lis2 = [round(val, 2) for val in lis]
print(lis2)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With