抓取产品信息，但没有得到想要的输出

Question

我试图在搜索结果中抓取产品的网页，首先尝试只抓取每个产品的标题:https://aiswatches.com/search.php?search_query_adv=16570&section=product

我知道我正在查看开发人员工具中html代码的正确部分，但我相信我的python代码缺少与<a标签有关的东西，我似乎无法获得语法来添加此:html代码>

下面是我的python代码:

import requests
from bs4 import BeautifulSoup
import pandas as pd
url = 'https://aiswatches.com/search.php?search_query_adv=16570&section=product'
def get_data(url):
r = requests.get(url)
soup = BeautifulSoup(r.text, 'html.parser')
return soup
def parse(soup):
results = soup.find_all('div', {'class': 'prodBox'})
for item in results:
products = {
'title': item.find('div', {'class': 'prodTitle'}).text
#'price':
#'link':
}
return
soup = get_data(url)
parse(soup)

任何帮助都将非常感激。我看了一个视频教程后卡住了。

Answer 1

我认为你的问题是,你不能得到一个链接的产品。link = watch.find('a').get('href')

url = 'https://aiswatches.com/search.php?search_query_adv=16570&section=product'
response = requests.get(url)
for watch in BeautifulSoup(response.text, 'html.parser').find_all('div', class_='prodTitle'):
title = watch.getText().strip()
link = watch.find('a').get('href')
price = watch.parent.findNext('div', class_='prodPrice').getText().strip()
print(title, price, link)

输出:

Rolex Polar Explorer II 16570 P Serial White Dial Box & Papers $10,975.00 https://aiswatches.com/rolex-polar-explorer-ii-16570-p-serial-white-dial-box-papers/
Rolex Polar Explorer II 16570 M Serial White Dial 3186 Movement Box & Papers $12,575.00 https://aiswatches.com/rolex-polar-explorer-ii-16570-m-serial-white-dial-3186-movement-box-papers/
Rolex Polar Explorer II 16570 P Serial White Dial Box & Papers + RSC Serviced $10,975.00 https://aiswatches.com/rolex-polar-explorer-ii-16570-p-serial-white-dial-box-papers-rsc-serviced-1/
Rolex Polar Explorer II 16570 Y Serial White Dial Box & Paper $10,975.00 https://aiswatches.com/rolex-polar-explorer-ii-16570-y-serial-white-dial-box-paper/
Rolex Polar Explorer II 216570 White Dial 42mm Random Serial B&P $11,975.00 https://aiswatches.com/rolex-polar-explorer-ii-216570-white-dial-42mm-random-serial-b-p-1/
Rolex Polar Explorer II 216570 White Dial 42mm Random Serial Box & Paper $12,975.00 https://aiswatches.com/rolex-polar-explorer-ii-216570-white-dial-42mm-random-serial-box-paper/
Rolex Explorer II 16570 M Serial Black Dial 3186 Movement $9,975.00 https://aiswatches.com/rolex-explorer-ii-16570-m-serial-black-dial-3186-movemen/