Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Spark - How to add a StructField at the beginning of a StructType in scala

I have a schema which is of type StructType:

val schema = getSchema();  // getSchema returns StructType

I created another field of type StructField:

StructField("NAME", StringType, false)

I know that I can call schema.add() method to add the NAME field to the end of the existing schema, but how do I add NAME to the beginning of the schema to make it the first column? Thanks.

like image 921
jlp Avatar asked Sep 05 '25 03:09

jlp


2 Answers

You can create an auxiliary StructType and then add your existing one to it:

val auxSchema=StructType(Array(StructField("NAME", StringType, false)))
StructType(auxSchema++schema)
like image 181
Christian Hirsch Avatar answered Sep 07 '25 19:09

Christian Hirsch


This will work.

 val schema = StructType(
      StructField("string_column", StringType, true) ::
        StructField("int_column", IntegerType, true) ::
        StructField("float_column", FloatType, true) ::
        StructField("_corrupt_record", StringType, true) :: Nil
    )

 val newschema = schema.add(StructField("new_column", StringType, true))
 println(newschema)
like image 42
Tufan Rakshit Avatar answered Sep 07 '25 20:09

Tufan Rakshit