apache-spark-sql tutorials

Speed up InMemoryFileIndex for Spark SQL job with large number of input files

Nov 12, 2022

apache-spark apache-spark-sql

Spark SQL: using collect_set over array values?

Jun 16, 2022

java apache-spark apache-spark-sql

Converting a list of rows to a PySpark dataframe

Aug 27, 2022

python apache-spark pyspark apache-spark-sql rows

How to normalize and create similarity matrix in Pyspark?

Sep 05, 2022

python pandas apache-spark pyspark apache-spark-sql

How to Access RDD Tables via Spark SQL as a JDBC Distributed Query Engine?

Sep 29, 2022

apache-spark apache-spark-sql

How to create a graph from Array[(Any, Any)] using Graph.fromEdgeTuples

Aug 30, 2022

scala apache-spark apache-spark-sql spark-graphx

`show tables like '*' fails in Spark SQL 1.3.0+

Oct 24, 2019

apache-spark-sql

DataFrame explode list of JSON objects

Oct 15, 2022

scala apache-spark apache-spark-sql distributed-computing

Memory issue when importing parquet files in Spark

Jan 24, 2022

scala apache-spark apache-spark-sql parquet

OneHotEncoder in Spark Dataframe in Pipeline

Aug 20, 2022

scala apache-spark apache-spark-sql apache-spark-mllib apache-spark-ml

How to avoid boxing bytes in array in custom datasource?

Dec 15, 2018

scala apache-spark apache-spark-sql

How to convert unix timestamp to the given timezone with Spark

Feb 16, 2022

scala apache-spark apache-spark-sql timezone unix-timestamp

Retain raw JSON as column in Spark DataFrame on read/load?

Aug 25, 2022

json apache-spark apache-spark-sql

Why do I get so many empty partitions when repartionning a Spark Dataframe?

Nov 18, 2022

apache-spark pyspark apache-spark-sql partitioning

NOT IN implementation of Presto v.s Spark SQL

Oct 26, 2022

null apache-spark-sql presto isnull

Spark SQL - Regex for matching only numbers

Nov 10, 2022

regex dataframe apache-spark pyspark apache-spark-sql

Spark window partition function taking forever to complete

Sep 14, 2022

scala performance dataframe apache-spark apache-spark-sql

How to compare multiple rows?

Nov 06, 2019

scala apache-spark spark-streaming apache-spark-sql

Using groupBy in Spark and getting back to a DataFrame

Nov 02, 2022

scala apache-spark apache-spark-sql

How to get date and time from string?

Dec 06, 2018

scala date apache-spark apache-spark-sql

New posts in apache-spark-sql