显示两个不同的错误，其代码与我用来抓取其他页面的代码相同

Question

我用一个代码从tripadvisor上抓取了两页，效果很好。但是现在，它向我展示了两个不同的错误：

with open("iletaitunsquare1.csv", "w", encoding="utf-8-sig", newline='') as csv_file:
w = csv.writer(csv_file, delimiter = ";", quoting=csv.QUOTE_MINIMAL)
w.writerow(["inf_rest_name", "rest_eclf", "name_client", "date_rev_cli", "opinion_cl"])
with requests. Session() as s:
for offset in range (270,1230,10):
url = f'https://www.tripadvisor.fr/Restaurant_Review-g187147-d6575305-Reviews-or{offset}-Il_Etait_Un_Square-Paris_Ile_de_France.html'
r = s.get(url)
soup = bs(r.content, 'lxml')
reviews = soup.select('.reviewSelector')
ids = [review.get('data.reviewid') for review in reviews]
r = s.post(
'https://www.tripadvisor.fr/OverlayWidgetAjax?Mode=EXPANDED_HOTEL_REVIEWS_RESP&metaReferer=',
data = {'reviews': ','.join(ids), 'contextChoice': 'DETAIL'},
headers = {'Referer': r.url}
)
soup = bs(r.content, 'lxml')
if not offset:
inf_rest_name = soup.select_one('.heading').text.replace("n","").strip()
rest_eclf = soup.select_one('.header_links a').text.strip()
for review in soup.select('.reviewSelector'):
name_client = review.select_one('.info_text > div:first-child').text.strip()
date_rev_cl = review.select_one('.ratingDate')['title'].strip()
titre_rev_cl = review.select_one('.noQuotes').text.strip()
opinion_cl = review.select_one('.partial_entry').text.replace("n","").strip()
row = [f"{inf_rest_name}", f"{rest_eclf}", f"{name_client}", f"{date_rev_cl}", f"{titre_rev_cl}", f"{opinion_cl}"]
w.writerow(row)

执行错误：

"data = {'reviews'： '，'.join(ids(， 'contextChoice'： 'DETAIL'} 类型错误：序列项 0：预期的 str 实例，找不到任何类型">

在我决定只更改第 6 行(网站页面(和第 7 行(Url(中的值之后：

with open("boutary.csv", "w", encoding="utf-8-sig", newline='') as csv_file:
w = csv.writer(csv_file, delimiter = ";", quoting=csv.QUOTE_MINIMAL)
w.writerow(["inf_rest_name", "rest_eclf", "name_client", "date_rev_cl", "titre_rev_cl", "opinion_cl"])
with requests.Session() as s:
for offset in range(40, 290, 10):
url = f'https://www.tripadvisor.fr/Restaurant_Review-g187147-d9783452-Reviews-or{offset}-Boutary-Paris_Ile_de_France.html'
r = s.get(url)
soup = bs(r.content, 'lxml')
reviews = soup.select('.reviewSelector')
ids = [review.get('data-reviewid') for review in reviews]
r = s.post(
'https://www.tripadvisor.fr/OverlayWidgetAjax?Mode=EXPANDED_HOTEL_REVIEWS_RESP&metaReferer=',
data = {'reviews': ','.join(ids), 'contextChoice': 'DETAIL'},
headers = {'referer': r.url}
)
soup = bs(r.content, 'lxml')
if not offset:
inf_rest_name = soup.select_one('.heading').text.replace("n","").strip()
rest_eclf = soup.select_one('.header_links a').text.strip()
for review in soup.select('.reviewSelector'):
name_client = review.select_one('.info_text > div:first-child').text.strip()
date_rev_cl = review.select_one('.ratingDate')['title'].strip()
titre_rev_cl = review.select_one('.noQuotes').text.strip()
opinion_cl = review.select_one('.partial_entry').text.replace("n","").strip()
row = [f"{inf_rest_name}", f"{rest_eclf}", f"{name_client}", f"{date_rev_cl}" , f"{titre_rev_cl}", f"{opinion_cl}"]
w.writerow(row)

它向我展示了

"row = [f"{inf_rest_name}"， f"{rest_eclf}"，

f"{name_client}"， f"{date_rev_cl}" ， f"{titre_rev_cl}"， f"{opinion_cl}"]

名称错误： 名称'inf_rest_name'未定义"

这些错误很奇怪，因为以前，我对其他URL使用相同的代码并且效果很好。 你能告诉我发生了什么吗？我怎样才能正确运行它？我将感谢您的帮助。

Answer 1

这是因为在原始代码中，未在此处发布，它依赖于偏移量 0 的真实/虚假值，在您之前的问题中，这是第一个偏移量。

例如，使用：

for offset in range(0, 10, 10):
if not offset:

第一个值 0 是 Falsy 与数字> 0(在本方案中(，将被视为 Truthy。如果不是 True，即 False，即如果偏移量为 0，则设置inf_rest_name的值。这可确保其值仅在第一个循环而不是每次设置。它的值不会改变，因此无需再次读取。

对于以下所有值都是真实值，因此inf_rest_name永远不会设置。

for offset in range(40, 290, 10):
if not offset:

您可以更改为：

if offset == firstvalue:

例如

if offset == 40:
inf_rest_name = soup.select_one('.heading').text.replace("n","").strip()
rest_eclf = soup.select_one('.header_links a').text.strip()

有关详细信息，请参阅此处。

这些行还需要处理第一个汤而不是后来的汤(因为这只是评论(

import requests
from bs4 import BeautifulSoup as bs
with requests.Session() as s:
for offset in range(40, 290, 10):
url = f'https://www.tripadvisor.fr/Restaurant_Review-g187147-d9783452-Reviews-or{offset}-Boutary-Paris_Ile_de_France.html'
r = s.get(url)
soup = bs(r.content, 'lxml')
if offset == 40:
inf_rest_name = soup.select_one('.heading').text.replace("n","").strip()
rest_eclf = soup.select_one('.header_links a').text.strip()
reviews = soup.select('.reviewSelector')
ids = [review.get('data-reviewid') for review in reviews]
r = s.post(
'https://www.tripadvisor.fr/OverlayWidgetAjax?Mode=EXPANDED_HOTEL_REVIEWS_RESP&metaReferer=',
data = {'reviews': ','.join(ids), 'contextChoice': 'DETAIL'},
headers = {'referer': r.url}
)
soup = bs(r.content, 'lxml')
for review in soup.select('.reviewSelector'):
name_client = review.select_one('.info_text > div:first-child').text.strip()
date_rev_cl = review.select_one('.ratingDate')['title'].strip()
titre_rev_cl = review.select_one('.noQuotes').text.strip()
opinion_cl = review.select_one('.partial_entry').text.replace("n","").strip()
row = [f"{inf_rest_name}", f"{rest_eclf}", f"{name_client}", f"{date_rev_cl}" , f"{titre_rev_cl}", f"{opinion_cl}"]

---

对于您的第一个代码块，您使用了无效的属性。它应该是

ids = [review.get('data-reviewid') for review in reviews]

注意 我添加了一个是无测试来处理未找到。这也应该添加到顶级版本中。

import requests
from bs4 import BeautifulSoup as bs
with requests. Session() as s:
for offset in range (270, 1230, 10):
url = f'https://www.tripadvisor.fr/Restaurant_Review-g187147-d6575305-Reviews-or{offset}-Il_Etait_Un_Square-Paris_Ile_de_France.html'
r = s.get(url)
soup = bs(r.content, 'lxml')
if offset == 270:
inf_rest_name = soup.select_one('.heading').text.replace("n","").strip()
rest_eclf = soup.select_one('.header_links a').text.strip()
reviews = soup.select('.reviewSelector')
ids = [review.get('data-reviewid') for review in reviews]
r = s.post(
'https://www.tripadvisor.fr/OverlayWidgetAjax?Mode=EXPANDED_HOTEL_REVIEWS_RESP&metaReferer=',
data = {'reviews': ','.join(ids), 'contextChoice': 'DETAIL'},
headers = {'Referer': r.url}
)
soup = bs(r.content, 'lxml')
for review in soup.select('.reviewSelector'):
name_client= review.select_one('.info_text > div:first-child')
if name_client is None:
name_client = 'N/A'
else:
name_client = name_client.text.strip()
date_rev_cl = review.select_one('.ratingDate')
if date_rev_cl is None:
date_rev_cl = 'N/A'
else:
date_rev_cl  = date_rev_cl['title'].strip()
titre_rev_cl = review.select_one('.noQuotes')
if titre_rev_cl is None:
titre_rev_cl = 'N/A'
else:
titre_rev_cl = titre_rev_cl.text.strip()
opinion_cl = review.select_one('.partial_entry')
if opinion_cl is None:
opinion_cl = 'N/A'
else:
opinion_cl =  opinion_cl.text.replace("n","").strip()
row = [f"{inf_rest_name}", f"{rest_eclf}", f"{name_client}", f"{date_rev_cl}", f"{titre_rev_cl}", f"{opinion_cl}"]
print(row)