尝试从一个站点中提取所有链接。我得到了";KeyError:'href'"但据我所知,这只适用于没有href的标签。然而,当我浏览汤对象时,每个a标记都有一个href。所以我不明白为什么我会看到这个错误。我查了很多,每个人都只提到一个没有href的标签。
from bs4 import BeautifulSoup
from datetime import datetime
import pandas as pd
import requests
page_count = 1
catalog_page = f"https://40kaudio.com/page/{str(page_count)}/?s"
while page_count < 4:
print(f"Begin Book Scrape from {catalog_page}")
# Soup opens the page.
open_page = requests.get(catalog_page)
# We create a soup object that has all the page stuff in it
soup = BeautifulSoup(open_page.content, "html.parser")
# We iterate through that soup object and pull out anything with a class of "title-post"
for link in soup.find_all('h2', "title-post"):
print(link['href'])
else:
print('By the Emperor!')
link
中没有href
标记。然而,在link
中有一个a
标签,而在a
标签中有href
属性。
<a href="https://40kaudio.com/justin-d-hill-cadia-stands-audiobook/" rel="bookmark">Justin D. Hill – Cadia Stands Audiobook</a>
for link in soup.find_all('h2', "title-post"):
print(link.a['href'])
不要忘记在while
循环中增加page_count
。