Pyspark DD-MMM-YYYY(字符串格式)到时间戳

Question

你好，我是Pyspark的新手，我有一个包含日期DD-MMM-YYYY格式的字符串变量，我想把它转换成时间戳吗？

2019年5月24日-字符串格式到时间戳

Answer 1

尝试to_timestamp(preferred) (or) from_unixtime and unix_timestamp函数：

Example:

from pyspark.sql.functions import *
from pyspark.sql.types import *
df.selectExpr("to_timestamp(dt,'dd-MMM-yyyy') as tt").show()
+-------------------+
|                 tt|
+-------------------+
|2019-05-24 00:00:00|
+-------------------+
df1.withColumn("ts",to_timestamp(col("dt"),'dd-MMM-yyyy')).show()
+-----------+-------------------+
|         dt|                 ts|
+-----------+-------------------+
|24-MAY-2019|2019-05-24 00:00:00|
+-----------+-------------------+
#using from_unixtime and unix_timestamp
df1.withColumn("ts",from_unixtime(unix_timestamp(col("dt"),'dd-MMM-yyyy'),'yyyy-MM-dd HH:mm:ss.SSS').cast("timestamp")).show(10,False)
+-----------+-----------------------+
|dt         |ts                     |
+-----------+-----------------------+
|24-MAY-2019|2019-05-24 00:00:00.000|
+-----------+-----------------------+
#using unix_timestamp and casting to timestamp
df1.withColumn("ts",unix_timestamp(col("dt"),'dd-MMM-yyyy').cast("timestamp")).show()
#+-----------+-------------------+
#|         dt|                 ts|
#+-----------+-------------------+
#|24-MAY-2019|2019-05-24 00:00:00|
#+-----------+-------------------+