Python BeautifulSoup网页抓取返回none

Question

from bs4 import BeautifulSoup
import requests
url = 'https://www.facilitron.com/searchfacility/u:facilitron/lat:37.8271784/lng:-122.2913078/dist:20/activities:-1/types:-1/namelike:grass%20field'
source = requests.get(url)
soup = BeautifulSoup(source.content, 'lxml')
find = soup.find('div', class_="search-results-body")
print(find)

我正在尝试获取给定网站的img、标题和位置，但BeautifulSoup无法使用类"获取div内部的元素；搜索结果主体"；当我试图找到这个div中包含类的div时，它返回none。

Answer 1

尝试将您的解析器更改为这个

soup = BeautifulSoup(source.content, 'html.parser')

Answer 2

您要查找的内容是高度动态的，并且不存在于页面源中(Ctrl+U(。然而，通过使用适当的参数发出post-http请求，您可以从一些json响应中获取它们。我只分析了每个潜在客户的姓名和地址。您可以从响应中找到与每条线索相关的许多其他信息。

import requests
url = 'https://www.facilitron.com/api/search/facilities/facilitron'
payload = {
'search_option[startLoc][]': ['-122.2913078','37.8271784'],
'search_option[distance]': '20',
'search_option[activities][]': '-1',
'search_option[start_row]': '0',
'search_option[limit]': '15',
'search_option[types][]': '-1',
'search_option[namelike]': 'grass field'
}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36',
'Referer': 'https://www.facilitron.com/searchfacility/u:facilitron/lat:37.8271784/lng:-122.2913078/dist:20/activities:-1/types:-1/namelike:grass%20field'
}
res = requests.post(url,data=payload,headers=headers)
for item in res.json()['facilities']:
print(item['name'],item['address'])

Answer 3

试试这个，

find = soup.find('div', {'class': 'clearfix'})