属性错误：尝试使用 writerow 时'str'对象没有属性'keys'

Question

尝试编写python scraper scrapes 数据从网页到CSV文件

如果我删除了dataframecleaned = cleandataup(dataframe(，则尝试更改我编写python文件的方式。CSVDATA(DataFrameCleaned(代码运行，但是它不将数据写入CSV文件

'''
write data to csv
'''
def csvData(dataFrame):
    with open('threads.csv', 'w+', newline='', encoding='utf8') as csvfile:
        fieldnames = ['post id', 'name', 'date of the post', 'post body']
        writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
        writer.writeheader()
        for posts in dataFrame:
            writer.writerow(posts)
        print('file write complete')

'''
defaults
'''
if __name__ == "__main__":
    path = str(os.path.dirname(os.path.realpath(__file__)))+'/data/'
    reload(sys)
    fieldnames = ['post id', 'name', 'date of the post', 'post body']
    dataFrame = pd.DataFrame(columns=fieldnames)
    url = 'http://www.oldclassiccar.co.uk/forum/phpbb/phpBB2/viewtopic.php?t=12591'
    urlList = [url]
    soup = get_soup(url)
    while True:
        newUrlSuffix = getURL(soup)
        if newUrlSuffix == '':
            break
        newUrl = 'http://www.oldclassiccar.co.uk/forum/phpbb/phpBB2/' + newUrlSuffix
        print("Adding new URL to list..")
        urlList.append(newUrl)
        soup = get_soup(newUrl)
    
    for link in urlList:
        print("Getting data from URL:" + link+ 'nnn')
        dataFrameNew = extractData(link)
        dataFrame = pd.concat([dataFrame,dataFrameNew])
    dataFrameCleaned = cleanDataUp(dataFrame)
    csvData(dataFrameCleaned)
The function for cleanDataUp
def cleanDataUp(dataFrame):
    dataFrame = dataFrame.reset_index(drop=True).dropna()
    return dataFrame

Traceback (most recent call last):
  File "scraper.py", line 127, in <module>
    csvData(dataFrameCleaned)
  File "scraper.py", line 96, in csvData
    writer.writerows(posts)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/csv.py", line 158, in writerows
    return self.writer.writerows(map(self._dict_to_list, rowdicts))
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/csv.py", line 148, in _dict_to_list
    wrong_fields = rowdict.keys() - self.fieldnames
AttributeError: 'str' object has no attribute 'keys'

Answer 1

在 writer.writerow(posts)中， writer是类型csv.dictwriter，该参数应该是字典，例如

writer.writerow({'first_name': 'Baked', 'last_name': 'Beans'})

但是，正如错误所说，posts是字符串而不是字典，因此错误 AttributeError: 'str' object has no attribute 'keys'

也许您的cleanDataUp(dataFrame)正在返回您的字符串列表，但是您需要词典列表，您需要检查该功能以确保其返回正确的输出以传递到csvData()函数