apache spark如何在sql表中获得不在另一个表中的列列表?

Question

表A有125列。表B有123列

select * from (desc A full outer join desc B on A.col_name = B.col_name) 

结果应该为空，对应于缺失的列。

为什么不工作并抛出错误mismatched input 'from' expecting <EOF>

Answer 1

DESCRIBE在FROM子句中不会像那样工作，相反，您可以实现您想要的结果，如下所示:

val df_a = spark.sql("desc table_1").select($"col_name" as "colA")
val df_b = spark.sql("desc table_2").select($"col_name" as "colB")
val schema_diff = df_a.join(df_b,df_a("colA")===df_b("colB"),"fullouter")
schema_diff.show()

输出:

+----+----+
|colA|colB|
+----+----+
|dept|null|
|  id|  id|
|name|name|
+----+----+