我试图在我的数据框架上应用一个枢轴
val pivot_company_model_vals_df = company_model_vals_df.groupBy("company_id","id","date")
.pivot("code")
.agg( when( col("data_item_value_numeric").isNotNull,
first("numeric")).otherwise(first("value")) )
错误
org.apache.spark.sql.AnalysisException: expression '`data_item_value_numeric`' is neither present in the group by, nor is it an aggregate function.
您能帮我我在这里做错了什么?谢谢
问题修复了移动first
的问题,如.agg( first(when
:
val pivot_company_model_vals_df = company_model_vals_df.groupBy("company_id","model_id","data_date")
.pivot("code")
.agg( first(when( col("data_item_value_numeric").isNotNull,
col("numeric")).otherwise(col("_string")) ) )