使用Beautiful soup查找公司名称和地址的电话号码

Question

我有一个脚本，可以在网站上抓取西班牙公司的名称、地区和省份。html中还有另一个链接，它会把你带到一个包含电话号码的页面，但当我试图抓取html时，它会打印出"没有"；。有没有一种方法可以让脚本自动移动到页面，抓取数字并将其与公司行匹配？

import requests
from googlesearch import search
from bs4 import BeautifulSoup
for page in range(1,65):
url = "https://www.expansion.com/empresas-de/ganaderia/granjas-en-general/{page}.html".format(page =page)
page = requests.get(url)
soup = BeautifulSoup(page.content, "html.parser")
lists = soup.select("div#simulacion_tabla ul")
#scrape the list
for lis in lists:
title = lis.find('li', class_="col1").text
location = lis.find('li', class_="col2").text
province = lis.find('li', class_="col3").text
link = lis.find('href', class_ ="col1")
info = [title, location, province, link]
print(info)

或者，有没有一种方法可以用谷歌搜索库做到这一点？

非常感谢

Answer 1

第一个url"https://www.expansion.com/empresas-de/ganaderia/granjas-en-general/index.html"而非

"https://www.expansion.com/empresas-de/ganaderia/granjas-en-general/1.html"

因此，您的脚本不会返回输出。

你可以试试这个

import requests
# from googlesearch import search
from bs4 import BeautifulSoup
baseurl = ["https://www.expansion.com/empresas-de/ganaderia/granjas-en-general/index.html"]
urls = [f'https://www.expansion.com/empresas-de/ganaderia/granjas-en-general/{i}.html'.format(i) for i in range(2,5)]

allurls = baseurl + urls
print(allurls)

for url in allurls:
page = requests.get(url)
soup = BeautifulSoup(page.content, "html.parser")
lists = soup.select("div#simulacion_tabla ul")
#scrape the list
for lis in lists:
title = lis.find('li', class_="col1").text
location = lis.find('li', class_="col2").text
province = lis.find('li', class_="col3").text
link = lis.select("li.col1 a")[0]['href']
info = [title, location, province, link]
print(info)