apache-spark-sql tutorials

Stream-Static Join: How to refresh (unpersist/persist) static Dataframe periodically

Sep 25, 2021

Spark DataFrame created from JavaRDD<Row> copies all columns data into first column

Sep 13, 2022

apache-spark apache-spark-sql

How is it possible to add new column to existing Dataframe in Spark SQL

May 11, 2022

java-8 dataframe apache-spark-sql spark-dataframe

Broadcast not happening while joining dataframes in Spark 1.6

Oct 20, 2022

scala apache-spark join apache-spark-sql query-optimization

How to drop rows with too many NULL values?

Mar 04, 2022

scala apache-spark dataframe apache-spark-sql

Pyspark : Custom window function

Jun 23, 2022

apache-spark pyspark apache-spark-sql window-functions

How to add new columns to DataFrame given their names when they are missing?

Jan 02, 2022

scala apache-spark dataframe apache-spark-sql

How to write rows asynchronously in Spark Streaming application to speed up batch execution?

Jun 27, 2022

performance apache-spark apache-spark-sql spark-streaming

spark-sql Table or view not found error

Jun 12, 2018

apache-spark apache-spark-sql spark-dataframe

How to join/merge a list of dataframes with common keys in PySpark?

Sep 28, 2022

python apache-spark pyspark apache-spark-sql

How to create schema (StructType) with one or more StructTypes?

Jun 23, 2022

scala apache-spark apache-spark-sql

PySpark aggregation function for "any value"

Oct 24, 2022

python apache-spark pyspark apache-spark-sql coalesce

Why does array_contains accept columns for both arguments in SQL but not in Dataset API?

Mar 12, 2022

apache-spark apache-spark-sql

Incompatible Jackson version: Spark Structured Streaming

Jun 18, 2022

scala apache-spark sbt apache-spark-sql

How to return rows with Null values in pyspark dataframe?

Oct 16, 2022

python pyspark apache-spark-sql

Number of dataframe partitions after sorting?

Oct 25, 2022

apache-spark apache-spark-sql

Drop rows containing specific value in PySpark dataframe

Sep 21, 2022

apache-spark pyspark apache-spark-sql pyspark-sql

Does Spark distributes dataframe across nodes internally?

Nov 13, 2022

apache-spark pyspark apache-spark-sql

spark 2.4.0 gives "Detected implicit cartesian product" exception for left join with empty right DF

Jun 17, 2022

apache-spark-sql

How to concatenate multiple columns in PySpark with a separator?

Sep 20, 2022

apache-spark pyspark apache-spark-sql

New posts in apache-spark-sql