获取文本内容时的美丽汤'Attribute Error'

Question

当我尝试使用BeautifulSoup时从我的web scraper打印出文本内容时，我得到了一个错误。这是我的代码

import requests
from bs4 import BeautifulSoup
form = "Form W-2"
URL = "https://apps.irs.gov/app/picklist/list/priorFormPublication." 
"html?resultsPerPage=200&sortColumn=sortOrder&indexOfFirstRow=0&criteria=formNumber&value=" 
""+form+"&isDescending=false"
page = requests.get(URL)
soup = BeautifulSoup(page.content, "html.parser")
results = soup.find(id="picklistContentPane")
# form_elements = results.find_all("div", class_="picklistTable")
table_elements = results.find_all("tr")
for table_element in table_elements:
form_number = table_element.find("td", class_="LeftCellSpacer")
form_title = table_element.find("td", class_="MiddleCellSpacer")
form_year = table_element.find("td", class_="EndCellSpacer")
print(form_number.text)
print(form_title.text)
print(form_year.text)
print()
# print(table_elements)

得到的错误是

File "/Users/user/PycharmProjects/project/main.py", line 18, in <module>
print(form_number.text)
AttributeError: 'NoneType' object has no attribute 'text'

我试图打印的文本内容是在每个td谁可以帮助?这是可供参考的网站

Answer 1

一个解决方案是选择正确的<table>，并且只选择包含<td>标签的行(<tr>):

import requests
from bs4 import BeautifulSoup
form = "Form W-2"
URL = (
"https://apps.irs.gov/app/picklist/list/priorFormPublication."
"html?resultsPerPage=200&sortColumn=sortOrder&indexOfFirstRow=0&criteria=formNumber&value="
"" + form + "&isDescending=false"
)
page = requests.get(URL)
soup = BeautifulSoup(page.content, "html.parser")
for table_element in soup.select(".picklist-dataTable tr:has(td)"):  # <-- change the selector here
form_number = table_element.find("td", class_="LeftCellSpacer")
form_title = table_element.find("td", class_="MiddleCellSpacer")
form_year = table_element.find("td", class_="EndCellSpacer")
print(form_number.text.strip())
print(form_title.text.strip())
print(form_year.text.strip())
print()

打印:

Form W-2 P
Statement For Recipients of Annuities, Pensions, Retired Pay, or IRA Payments
1990
Form W-2 P
Statement For Recipients of Annuities, Pensions, Retired Pay, or IRA Payments
1989
Form W-2 P
Statement For Recipients of Annuities, Pensions, Retired Pay, or IRA Payments
1988

...and so on.