使用蟒蛇美丽汤进行网络爬行

Question

如何提取<p>段落标签内的数据，以及命名<div>类下<li>？

Answer 1

使用函数find()和find_all()：

import requests
from bs4 import BeautifulSoup
url = '...'
r = requests.get(url)
data = r.text
soup = BeautifulSoup(data, 'html.parser')
div = soup.find('div', {'class':'class-name'})
ps = div.find_all('p')
lis = div.find_all('li')
# print the content of all <p> tags
for p in ps:
    print(p.text)
# print the content of all <li> tags
for li in lis:
    print(li.text)