我一直在尝试获得那种"结果显示页面"的第一个结果的url;我不能，html解析器不包含它…
网站链接:https://www.sobrico.com/#Prod_Live_Sobrico%5Bquery%5D=2608664131
我一直在尝试使用BeautifulSoup，请求不同的代码，但没有任何结果。当我在像这样的产品页面上时，我能够抓取许多信息:https://www.sobrico.com/p/bosch-2608664131-coffret-lames-best-for-cutting-bosch-2608664131_SKU726760.html
但是在搜索结果页面上，代码的某些部分，首先是包含显示结果的部分，是不可用的。我希望能得到一个答案，那会很有帮助的。
下面是我的代码:import requests
from bs4 import BeautifulSoup
URL = "https://www.sobrico.com/#Prod_Live_Sobrico%5Bquery%5D=2608664131"
page = requests.get(URL)
soup = BeautifulSoup(page.content, features='lxml')
for link in soup("a"):
print(link.get("href"))
您可以使用Selenium来获取第一项在搜索结果页面上。使用CSS选择器来标识感兴趣的项。然后使用get_attribute()函数获取url。
代码:
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get("https://www.sobrico.com/#Prod_Live_Sobrico%5Bquery%5D=2608664131")
first_item = driver.find_element(By.CSS_SELECTOR, "div#algoliasearch-hits li:nth-child(1) > article > a")
url = first_item.get_attribute("href")
print(url)
driver.close()
https://www.sobrico.com/p/bosch-2608664131-coffret-lames-best-for-cutting-bosch-2608664131_SKU726760.html
注意:
如果您想从页面上的11个条目中获取url，您可以遍历它们:
for i in range(1,12):
url = driver.find_element(By.CSS_SELECTOR, f"div#algoliasearch-hits li:nth-child({i}) > article > a").get_attribute("href")
print(url)
https://www.sobrico.com/p/bosch-2608664131-coffret-lames-best-for-cutting-bosch-2608664131_SKU726760.html
https://www.sobrico.com/p/bosch-coffret-starlockmax-4-pieces-bosch-best-of-heavy-duty-2608664132_SKUM89146.html
https://www.sobrico.com/p/bosch-expert-2608644113-lame-scie-circulaire-260x30x2-8-mm-aluminium-96-dents-bosch-expert-2608644113_SKU744566.html
https://www.sobrico.com/p/bosch-2608631319-5-lames-scie-sauteuse-hss-132-mm-metal-t318a-bosch-2608631319_SKU478349.html
https://www.sobrico.com/p/bosch-2608641185-lame-scie-circulaire-190x30x2mm-bois-24-dents-bosch-2608641185_SKU491105.html
https://www.sobrico.com/p/bosch-2608636431-lames-scie-sauteuse-t101-bif-special-for-laminate-pack-5-bosch-2608636431_SKU478354.html
https://www.sobrico.com/p/bosch-expert-2608606131-bande-abrasive-ponceuses-x440-best-for-wood-and-paint-grain-80-610mm-bosch-2608606131_SKU490187.html
https://www.sobrico.com/p/bosch-lames-scie-circulaires-bosch-construct-wood_SKUM75099.html
https://www.sobrico.com/p/bosch-expert-2608644134-lames-scie-circulaire-for-high-pressure-laminate-56-dents-bosch-expert-2608644134_SKU744579.html
https://www.sobrico.com/p/bosch-2608662431-lame-scie-oscillante-asz-32-sc-outils-multi-fonctions-bosch-2608662431_SKU777336.html
https://www.sobrico.com/p/bosch-2608664348-lame-scie-oscillante-bosch-starlock-carbure-metalmax-aiz-45-at-45mm-bosch-expert-2608664348_SKU777339.html