apache-spark-sql tutorials

Scala - How to split the probability column (column of vectors) that we obtain when we fit the GMM model to the data in to two separate columns? [duplicate]

Aug 31, 2022

How does Spark SQL read compressed csv files?

Sep 14, 2022

csv apache-spark apache-spark-sql

reuse the result of a select expression in the "GROUP BY" clause?

Apr 05, 2021

mysql scala apache-spark apache-spark-sql spark-dataframe

Pyspark Dataframe - Map Strings to Numerics

Oct 20, 2022

apache-spark pyspark apache-spark-sql spark-dataframe pyspark-sql

How to calculate the power of 2 for the column of DataFrame

Jul 04, 2022

scala apache-spark apache-spark-sql

why does spark appends 'WHERE 1=0' at the end of sql query

Nov 10, 2022

apache-spark apache-spark-sql spark-dataframe

Save the parquet output file with fixed size in spark

Oct 01, 2022

apache-spark apache-spark-sql

Spark's .count() function is different to the contents of the dataframe when filtering on corrupt record field

Feb 06, 2022

apache-spark pyspark apache-spark-sql

How do I groupby and concat a list in a Dataframe Spark Scala

Nov 08, 2022

scala apache-spark dataframe apache-spark-sql

Spark & Scala: saveAsTextFile() exception

Oct 22, 2022

scala apache-spark hadoop apache-spark-sql bigdata

contains pyspark SQL: TypeError: 'Column' object is not callable

Apr 25, 2022

python apache-spark pyspark apache-spark-sql

How to show my existing column name instead '_c0', '_c1', '_c2', '_c3', '_c4' in first row?

Sep 05, 2022

pyspark apache-spark-sql azure-databricks spark-notebook

Spark Parquet read error : java.io.EOFException: Reached the end of stream with XXXXX bytes left to read

Jul 19, 2022

apache-spark apache-spark-sql parquet

Using pyspark, how to expand a column containing a variable map to new columns in a DataFrame while keeping other columns?

Jun 22, 2022

apache-spark pyspark apache-spark-sql

Pyspark filter dataframe if column does not contain string

Nov 03, 2022

python apache-spark pyspark apache-spark-sql

Weird behaviour with spark-submit

Apr 19, 2022

apache-spark hive cloudera-cdh apache-spark-sql

How does Spark DataFrame handles Pandas DataFrame that is larger than memory

May 28, 2022

pandas apache-spark dataframe apache-spark-sql hdf5

java.lang.UnsupportedOperationException: 'Writing to a non-empty Cassandra Table is not allowed

Jun 15, 2022

apache-spark cassandra apache-spark-sql spark-streaming datastax-enterprise

How to convert DataFrame columns from string to float/double in PySpark 1.6?

Mar 20, 2021

python pyspark apache-spark-sql type-conversion

How to select constant values from Dataframe coding in Java

Jan 15, 2018

java apache-spark dataframe apache-spark-sql bigdata

New posts in apache-spark-sql