pyspark 2.2 'DataFrame'对象没有属性'map'，向后兼容缺失如何解决

Question

当我使用Spark 1.6时，代码正常工作：

ddl = sqlContext.sql("""show create table {mytable }""".format(mytable="""mytest.my_dummytable"""))
map(''.join, ddl
.map(lambda my_row: [str(data).replace("`", "'") for data in my_row])
.collect())

但是，当我移动到Spark 2.2时，我得到了以下例外：

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-> in <module>()
      1 ddl = sqlContext.sql("""show create table {mytable }""".format(mytable ="""mytest.my_dummytable"""))
----> 2 map(''.join, ddl..map(lambda my_row: [str(data).replace("`", "'") for data in my_row]).collect())
spark2/python/pyspark/sql/dataframe.py in __getattr__(self, name)
            if name not in self.columns:
                raise AttributeError(
->                  "'%s' object has no attribute '%s'" % (self.__class__.__name__, name))
            jc = self._jdf.apply(name)
            return Column(jc)
AttributeError: 'DataFrame' object has no attribute 'map'

Answer 1

您必须先致电.rdd。SPARK 2.0停止将df.map()列为df.rdd.map()。看到此。