无法捕获请求Python中的记录名称、价格、评级和图像

Question

打印产品名、产品尺寸、价格和评级时发生异常这是我想从中提取细节的链接。

import requests
import time
from requests.models import Response

params = ((
'url','/continental-80-shoes/G27707.html'),
('sitePath', 'us'),)


headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36'
}
response = requests.get('https://www.adidas.com/api/metadata/pdp',params=params,headers=headers)
for item in response.json()['metadata']:
itemRes = requests.get('https://www.adidas.com/api/search/product/'+item['productId'],headers=headers) 
print(item['productId'],item['name'],item['price'],item['rating])

Answer 1

你必须刮掉adidas网站并使用regex:

import requests
import re
endpoint = "https://www.adidas.com.au/continental-80-shoes/G27707.html"
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36'
}
response = requests.get(endpoint, headers = headers)
data = response.text
pricereg = r"(?<="price":)(.*)(?=,")"
namereg = r"(?<="name":)(.*)(?=,"co)"
ratingreg= r"(?<="ratingValue":)(.*)(?=,"reviewCou)"
price = re.search(pricereg, data, re.MULTILINE).group()
name = re.search(namereg, data, re.MULTILINE).group()
rating = re.search(ratingreg, data, re.MULTILINE).group()
print(f"name {name}, rating {rating}, price {price}")