How can I create Dataset using StructType
?
We can create a Dataset
as follows:
case class Person(name: String, age: Int)
val personDS = Seq(Person("Max", 33), Person("Adam", 32), Person("Muller",
62)).toDS()
personDS.show()
Is there a way to create a Dataset
without using a case class?
I'd like to create a DataFrame
using a case class and using StructType
.
If you know how to create DataFrame, you already now how to create Dataset :)
DataFrame = Dataset[Row].
What it means? Try:
val df : DataFrame = spark.createDataFrame(...) // with StructType
import org.apache.spark.sql._
val ds : Dataset[Row] = df; // no error, as DataFrame is only a type alias of Dataset[Row]
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With