Integertype() in schema StructType

Question

I am trying to define a schema to convert a blank list into dataframe as per syntax below:

data=[]

schema = StructType([
StructField("Table_Flag",StringType(),True),
StructField("TableID",IntegerType(),False),
StructField("Entry_No",StringType(),True),
StructField("Dimension_Code",StringType(),True),
StructField("Dimension_Value_Code",StringType(),True)
])

data=spark.createDataFrame(data,schema)

but it is giving a error of name 'IntegerType' is not defined. I have tried with BooleanType, DecimalType, FloatType,IntegralType but none is working. Only StringType and DataType is available as datatypes. As per documentation types.py IntegerType is defined in examples.

Please suggest. I am working on spark version 2.0.1

Answer 1

Add import at the beginning of the file:

from pyspark.sql.types import IntegerType

Or even simpler:

from pyspark.sql.types import *

To import all classes from pyspark.sql.types