如何处理网页中的不同元素(国家天气)进行网页抓取

Question

import requests
from bs4 import BeautifulSoup
http="https://forecast.weather.gov/MapClick.php?lat=34.05361000000005&lon=-118.24549999999999#.X06DsYtxXIU"
response=requests.get(http).text
soup=BeautifulSoup(response,"html.parser")

for questiones in soup.find_all(class_='tombstone-container'):
tag=questiones.find('p',class_='period-name').text
print(tag)
sit=questiones.find('p',class_="short-desc").text
print(sit)
temp = questiones.find('p', class_='temp temp-low').text
print(temp)
if temp ==True:
print(temp)
else:
high=questiones.find('p',class_="temp temp-high").text
print(high)

Answer 1

如果元素不存在，则您正在尝试访问NoneType中的text属性。你应该以某种方式检查你是否找到了这样的元素：

import requests
from bs4 import BeautifulSoup
http="https://forecast.weather.gov/MapClick.php?lat=34.05361000000005&lon=-118.24549999999999#.X06DsYtxXIU"
response=requests.get(http).text
soup=BeautifulSoup(response,"html.parser")

for questiones in soup.find_all(class_='tombstone-container'):
tag=questiones.find('p',class_='period-name')
if tag:
print(tag.text)
sit=questiones.find('p',class_="short-desc")
if sit:
print(sit.text)
temp = questiones.find('p', class_='temp temp-low')
if temp:
print(temp.text)
high=questiones.find('p',class_="temp temp-high")
if high:
print(high.text)

此打印：

Overnight
Patchy Fog
Low: 63 °F
Wednesday
Patchy Fogthen Sunny
High: 82 °F
WednesdayNight
Patchy Fog
Low: 63 °F
Thursday
Patchy Fogthen Sunny
High: 86 °F
ThursdayNight
Clear
Low: 65 °F
Friday
Sunny
High: 90 °F
FridayNight
Clear
Low: 73 °F
Saturday
Hot
High: 103 °F
SaturdayNight
Clear
Low: 78 °F