apache-spark-sql tutorials

PySpark How to read CSV into Dataframe, and manipulate it

May 29, 2019

How to spark-submit a python file in spark 2.1.0?

Dec 04, 2019

apache-spark pyspark apache-spark-sql pyspark-sql spark-submit

PySpark: How to judge column type of dataframe

Mar 10, 2022

python apache-spark pyspark apache-spark-sql pyspark-sql

How to get table names from SQL query?

Aug 31, 2022

scala apache-spark apache-spark-sql

Clone/Deep-Copy a Spark DataFrame

Aug 20, 2022

scala apache-spark apache-spark-sql

How to handle white spaces in dataframe column names in spark

Sep 09, 2022

apache-spark pyspark apache-spark-sql

org.apache.spark.rpc.RpcTimeoutException: Futures timed out after [120 seconds]. This timeout is controlled by spark.rpc.lookupTimeout

Aug 16, 2017

apache-spark apache-spark-sql hadoop-yarn hadoop2

How to get data of previous row in Apache Spark

Mar 30, 2022

dataframe scala apache-spark apache-spark-sql

Compare Value of Current and Previous Row in Spark

Oct 22, 2022

scala apache-spark apache-spark-sql

Convert string to timestamp for Spark using Scala

Nov 05, 2022

scala apache-spark apache-spark-sql timestamp

How to use Column.isin in Java?

Nov 03, 2022

java apache-spark apache-spark-sql

How to do mathematical operation with two column in dataframe using pyspark

Aug 25, 2018

apache-spark pyspark apache-spark-sql spark-dataframe pyspark-sql

Prepend zeros to a value in PySpark

Oct 15, 2020

sql apache-spark pyspark apache-spark-sql

How to do conditional "withColumn" in a Spark dataframe?

Jan 30, 2019

scala apache-spark apache-spark-sql

How to execute .sql file in spark using python

Nov 16, 2022

python apache-spark apache-spark-sql pyspark

Write spark dataframe to postgres Database

Nov 20, 2022

postgresql apache-spark dataframe apache-spark-sql

Can I read multiple files into a Spark Dataframe from S3, passing over nonexistent ones?

May 27, 2022

python apache-spark pyspark apache-spark-sql pyspark-sql

How to concatenate multiple columns into single column (with no prior knowledge on their number)?

Jul 18, 2019

scala apache-spark apache-spark-sql

How to get the value of the location for a Hive table using a Spark object?

Feb 04, 2022

apache-spark hive apache-spark-sql

What is the right Date/Datetime format in JSON for Spark SQL to automatically infer the schema for it?

Nov 18, 2022

apache-spark apache-spark-sql

New posts in apache-spark-sql