抓取网站不断返回'NoneType'对象没有属性'find'错误 (BS4)

Question

我是BS4的新手。所以我很难理解为什么这个错误一直出现。我想查找图书的名称、排名、作者、评级和价格，但每次运行代码时，它们都返回该错误。

下面是我的代码:

try:
url = requests.get("https://www.amazon.in/gp/bestsellers/books/1318158031/ref=zg_bs_nav_books_1")
url.raise_for_status()
soup = BeautifulSoup(url.text, "html.parser")
books = soup.find("div", class_="p13n-gridRow _cDEzb_grid-row_3Cywl").find_all("div", id="gridItemRoot")

for book in books:
rank = book.find("div", class_="aok-float-left").span.text.split("#")[1]
name = book.find("div", class_="zg-grid-general-faceout").span.div.text
author = book.find("div", class_="a-row a-size-small").div.text
rating = book.find("div", class_="a-icon-row").find("a", class_="a-link-normal")(["title"])
price = book.find("div", class_="a-row").next_sibling.next_sibling.next_sibling.span.text
print(name)


except Exception as e:
print(e)

Answer 1

您正在获得NoneType error，因为清单/html树中缺少一些项目。因此，为了消除这种错误，您可以使用if else None语句

import pandas as pd
import requests
from bs4 import BeautifulSoup
headers={'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36'}
url='https://www.amazon.in/gp/bestsellers/books/1318158031/ref=zg_bs_nav_books_1'
req = requests.get(url,headers=headers)
print(req)
soup = BeautifulSoup(req.text, 'lxml')
books = soup.find("div", class_="p13n-gridRow _cDEzb_grid-row_3Cywl").find_all("div", id="gridItemRoot")

for book in books:
rank = book.find("div", class_="aok-float-left").span.text.split("#")[1]
name = book.find("div", class_="zg-grid-general-faceout").span.div
name=name.text if name else None
author = book.find("div", class_="a-row a-size-small").div.text
rating = book.find("span", class_="a-icon-alt")
rating=rating.text if rating else None
price = book.select_one(".a-size-base.a-color-price span").text
#price=price.text if price else None
print(price)

输出:

₹299.00
₹316.00
₹139.00
₹323.00
₹284.05
₹761.00
₹187.00
₹299.00
₹222.30
₹299.00
₹299.00
₹550.00
₹139.00
₹305.99
₹299.00
₹256.00
₹297.00
₹1,012.00
₹309.00
₹109.00
₹399.00
₹292.74
₹289.75
₹410.00
₹279.30
₹125.00
₹313.95
₹449.00
₹357.00
₹198.00