R POSIXlt timestamp conversion do not know how to convert 'df1$timestamp' to class “POSIXlt”

Question

Hello all I am facing issue while converting timestamp to POSIXlt, later from this timestamp o need to extract Year,month,dayofmoth,hour,min,sec

    2015-12-01 00:04:39 is my timestamp

and here is my try

getwd()
rm(list=ls())
library(ggplot2)
library(plyr)
library(reshape)
library(scales)
library(gridExtra)
library(SparkR)

Sys.setenv(SPARK_HOME="/usr/local/spark").libPaths(c(file.path(Sys.getenv("SPARK_HOME"), "R", "lib"), .libPaths()))

sc <- sparkR.init(master="local","RwordCount")
args <- commandArgs(trailing = TRUE)

sqlContext <- sparkRSQL.init(sc)

df1 <- read.df(sqlContext, "hdfs://master:9000/test.csv",  header='true', source = "com.databricks.spark.csv", inferSchema='true', stringsAsFactors = F)


 if("timestamp" %in% colnames(df1)){
     df1$pTime <- as.POSIXlt(df1$timestamp, format= "%Y-%m-%d  %H:%M:%S") 
  }else {
     df1$pTime <- as.POSIXlt(df1$Timestamp, format= "%Y-%m-%d %H:%M:%S")
}

but getting error here do not know how to convert 'df1$timestamp' to class “POSIXlt”

later i need to find out the Year,month,dayofmonth,hors,min,sec for that i have this snippet

df1$Year <- df1$pTime$year-100  #Year
df1$Month <- df1$pTime$mon+1  #Month 1-12
df1$Day <-  df1$pTime$mday  #day of month
df1$Hour <- df1$pTime$hour  #0â??23: hours
df1$Min <- df1$pTime$min
df1$Sec <- df1$pTime$sec
df1$WeekOfYear <- strftime(df1$pTime, format="%W")

and i am executing above script using following syntax,

 bin/spark-submit --packages com.databricks:spark-csv_2.11:1.3.0  /home/script/analysis.R 

 **Error in as.POSIXlt.default(df1$timestamp, format = "%Y-%m-%d %H:%M:%S") : 
  do not know how to convert 'df1$timestamp' to class “POSIXlt”
  Calls: as.POSIXlt -> as.POSIXlt.default
  Execution halted**

How can i get rid of the error, any help will be appreciated. Thanks

Answer 1

You can convert your timestamp by using as.POSIXct

x <- as.POSIXct("2015-12-01 00:04:39")

and then using lubridate package, you can extract all the information

library(lubridate)

year(x)
#[1] 2015
month(x)
#[1] 12
day(x)
#[1] 1
hour(x)
#[1] 0
minute(x)
#[1] 4
second(x)
#[1] 39

Answer 2

You can extract parts of datetime values by

x <- Sys.time()
format(x, format="%Y")

for example. See

?strptime

for all options.

I can not reconstruct the first part of your question. What is the error message you get?