无法使用美丽汤从隐藏的标签中抓取'value'

Question

我试图刮£6,550的基本价格从这个链接:https://www.plates4less.co.uk/private-plate_o/CSG1S它是在一个隐藏的输入称为BasePrice，当我查看页面源'6550'是在一个属性称为value:

<input type="hidden" name="BasePrice" id="BasePrice" value="6550">

我的代码是:

url = 'https://www.plates4less.co.uk/private-plate_o/CSG1S'
req = session.get(url, headers=headers)
bsObj = BeautifulSoup(req.text, features="html.parser")
price = bsObj.find("input", {"name" : {"BasePrice"}})
print(price['value'])

但是它返回KeyError: 'value'

当我运行这个(即检索标签，但没有尝试访问'value':

url = 'https://www.plates4less.co.uk/private-plate_o/CSG1S'
req = session.get(url, headers=headers)
bsObj = BeautifulSoup(req.text, features="html.parser")
price = bsObj.find("input", {"name" : {"BasePrice"}})
print(price)

我得到这个:

<input id="BasePrice" name="BasePrice" type="hidden"/>

为什么我不能访问value属性?

Answer 1

您在页面上看到的数据是从外部URL加载的。要加载基本价格，您可以使用以下示例:

import json
import requests
# CSG1S is from your base URL
url = "https://www.plates4less.co.uk/apiv3/TransferOptions/CSG1S"
data = requests.get(url).json()
# uncomment to see all data:
# print(json.dumps(data, indent=4))
print(data["data"]["registrationNumber"]["displayedBPrice"])

打印:

6550.0