我正在尝试制作一个浏览多个页面的webscraper,为我制作csv列表。当我运行基本代码时,它是有效的,但当我让它迭代到多个页面时,我会得到AttributeError:
assignees=assignees_elem.text.strip((AttributeError:"NoneType"对象没有属性"text">
csv已保存,但为空。
这是我的代码:
import requests
from bs4 import *
import csv
pagenumber = 0
file = open('newcsv', 'w')
writer = csv.writer(file)
while pagenumber <50:
pagenumber += 1
pagenumbers = str(pagenumber)
URL = 'website'+pagenumbers
page = requests.get(URL)
soup = BeautifulSoup(page.content, 'html.parser')
site_elems = soup.find_all('div', class_='meta')
writer.writerow(['Number', 'Issued', 'Assignees', 'Inventors'])
for site_elem in site_elems:
number_elem = site_elem.find('div', class_='number')
issued_elem=site_elem.find('div', class_='date-issued')
assignees_elem=site_elem.find('div', class_='assignees')
inventors_elem=site_elem.find('div', class_='inventors')
number=number_elem.text.strip()
issued=issued_elem.text.strip()
assignees=assignees_elem.text.strip()
inventors=inventors_elem.text.strip()
print (number + ' ' + issued +' ' + assignees + ' ' + inventors)
writer.writerow([number.encode('utf-8'),issued.encode('utf-8'), assignees.encode('utf-8'),inventors.encode('utf-8')])
file.close()
我通过添加解决了这个问题
if None in (number_elem, issued_elem, assignees_elem, inventors_elem): continue