Transpose rows to columns in pyspark

Question

How can I transpose a Dataframe table with only one column and multiple rows like:

1
2
3
5
6
7
...

to a dataframe with only one row and multiple columns like:

1,2,3,4,5,6,7,8,9,10,...

Answer 1

Simply do a pivot:

df = spark.range(10)

df.show()
+---+                                                                           
| id|
+---+
|  0|
|  1|
|  2|
|  3|
|  4|
|  5|
|  6|
|  7|
|  8|
|  9|
+---+


df.groupBy().pivot("id").count().show()                                                                            
+---+---+---+---+---+---+---+---+---+---+                                       
|  0|  1|  2|  3|  4|  5|  6|  7|  8|  9|
+---+---+---+---+---+---+---+---+---+---+
|  1|  1|  1|  1|  1|  1|  1|  1|  1|  1|
+---+---+---+---+---+---+---+---+---+---+

df.groupBy().pivot("id").agg(F.first(F.col("id"))).show()                                                          
+---+---+---+---+---+---+---+---+---+---+                                       
|  0|  1|  2|  3|  4|  5|  6|  7|  8|  9|
+---+---+---+---+---+---+---+---+---+---+
|  0|  1|  2|  3|  4|  5|  6|  7|  8|  9|
+---+---+---+---+---+---+---+---+---+---+