如何在连接后选择和排序Pyspark Dataframe中的多个列

Question

我想从现有的数据框架中选择多个列(这是在连接后创建的)，并希望将这些字段排序为我的目标表结构。怎样才能做到呢?我使用的方法如下。在这里，我可以选择所需的必要列，但不能按顺序制作。

Required (Target Table structure) :
hist_columns = ("acct_nbr","account_sk_id", "zip_code","primary_state", "eff_start_date" ,"eff_end_date","eff_flag")
account_sk_df = hist_process_df.join(broadcast(df_sk_lkp) ,'acct_nbr','inner' )
account_sk_df_ld = account_sk_df.select([c for c in account_sk_df.columns if c in hist_columns])
>>> account_sk_df
DataFrame[acct_nbr: string, primary_state: string, zip_code: string, eff_start_date: string, eff_end_date: string, eff_flag: string, hash_sk_id: string, account_sk_id: int]

>>> account_sk_df_ld
DataFrame[acct_nbr: string, primary_state: string, zip_code: string, eff_start_date: string, eff_end_date: string, eff_flag: string, account_sk_id: int]

account_sk_id需要放在第二位。最好的方法是什么?

Answer 1

尝试通过给出一个列表来选择列，而不是通过迭代现有的列，排序应该是OK的:

account_sk_df_ld = account_sk_df.select(*hist_columns)