Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Spark : how to create a row with fields name

I'm some tests, I need to create a Row and get its values using field name, however from the doc I can only create a Row using values, here is an example :

val row:Row=Row("aa","bb","cc")
//when I a try to get a field :
row.getAs("aa") 
I get : fieldIndex on a Row without schema is undefined

here is what I want :

//some way to add fields name
val row:Row=Row({aa:"aa",bb:"bb",cc:"cc"})
row.getAs("aa") //returns "aa"

I wonder if there is a better way, other than creating a dataframe and getting the Row from it

like image 684
aName Avatar asked Dec 14 '25 05:12

aName


1 Answers

You can create a Row with a defined schema:

val schema = StructType(Array(
  StructField("aa", StringType),
  StructField("bb", StringType),
  StructField("cc", StringType)
))

val row = new GenericRowWithSchema(Array("AA", "BB", "CC"), schema)

println(row.getAs[String]("aa"))
like image 55
Qin Avatar answered Dec 16 '25 19:12

Qin



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!