如何在PYSPARK中实现每月行数变化时的每月滞后?

Question

我想创建一个月滞后，以便计算月回报。

这是我的数据框架:

412316860416 420906795008429496729600438086664192446676598784455266533376463856467968472446402560481036337152489626271744498216206336506806140928515396075520523986010112532575944704541165879296549755813888558345748480566935683072575525617664584115552256592705486848601295421440609885356032618475290624627065225216635655159808644245094400652835028992661424963584670014898176

您可以通过在滞后窗口中设置分区来获得每种类型的前一行:

from pyspark.sql import functions as F, Window
df2 = df.withColumn(
'lag_production', 
F.lag('production').over(
Window.partitionBy('TYPE_CD')
.orderBy('REG_DT_YYYYMM')
)
)