apache-spark-sql tutorials

How to get date and time from string?

Dec 06, 2018

pyspark expected zero arguments for construction of ClassDict (for pyspark.mllib.linalg.DenseVector)

Dec 09, 2021

apache-spark pyspark apache-spark-sql user-defined-functions apache-spark-mllib

create hive external table with schema in spark

Nov 14, 2021

apache-spark hive apache-spark-sql spark-avro

How to GROUPING SETS as operator/method on Dataset?

Sep 10, 2022

apache-spark dataframe apache-spark-sql

PySpark: Get first Non-null value of each column in dataframe

Nov 03, 2022

python apache-spark dataframe pyspark apache-spark-sql

How to fill none values with a concrete timestamp in DataFrame?

Apr 22, 2022

apache-spark pyspark apache-spark-sql

PySpark - Compare DataFrames

Feb 15, 2022

python dataframe apache-spark pyspark apache-spark-sql

Processing multiple files as independent RDD's in parallel

May 09, 2022

scala apache-spark apache-spark-sql

Joining PySpark DataFrames on nested field

Oct 28, 2022

apache-spark dataframe join pyspark apache-spark-sql

How to ensure partitioning induced by Spark DataFrame join?

Jun 25, 2022

apache-spark dataframe join pyspark apache-spark-sql

Spark write to postgres slow

Oct 20, 2022

apache-spark dataframe apache-spark-sql

Peak Execution Memory in Spark

May 18, 2022

apache-spark apache-spark-sql

Find median in spark SQL for multiple double datatype columns

Oct 15, 2022

apache-spark apache-spark-sql hive-udf

Apache spark case with multiple when clauses on different columns

Jun 02, 2022

apache-spark hadoop apache-spark-sql

How to load a csv directly into a Spark Dataset?

Oct 23, 2022

scala apache-spark apache-spark-sql

merge two dataset which are having different column names in Apache spark

Aug 23, 2022

java apache-spark apache-spark-sql spark-dataframe

Why does spark-shell fail with "The root scratch dir: /tmp/hive on HDFS should be writable."?

Jul 31, 2022

apache-spark windows-10 apache-spark-sql

Why does a query fail with "AnalysisException: Expected only partition pruning predicates"?

May 15, 2019

apache-spark apache-spark-sql

What type should it be , after using .toArray() for a Spark vector?

Sep 10, 2022

python numpy apache-spark pyspark apache-spark-sql

Self-join not working as expected with the DataFrame API

Sep 05, 2022

scala apache-spark dataframe apache-spark-sql

New posts in apache-spark-sql