解析div中的json对象,同时使用beautifulsoup-python进行抓取



我正在学习刮削。我需要访问我在DIV中遇到的json字符串。我使用的是beautifulsoup。这是我在DIV中得到的json字符串。我需要标签"lastprice"的值(51.65(。请帮忙。JSON对象在JSON_d 中

import pip
import requests
import json 
from bs4 import BeautifulSoup
print ('hi')
page = requests.get('https://www.nseindia.com/live_market/dynaContent/live_watch/get_quote/GetQuote.jsp?symbol=NBCC&illiquid=0&smeFlag=0&itpFlag=0')
soup = BeautifulSoup(page.text, 'html.parser')
json_d = soup.find(id='responseDiv')
print ('bye')
import bs4
import json
r= '''
<div id="responseDiv" style="display:none">{"tradedDate":"07DEC2018","data":[{"pricebandupper":"58.35","symbol":"NBCC","applicableMargin":"15.35","bcEndDate":"14-SEP-18","totalSellQuantity":"40,722","adhocMargin":"-","companyName":"NBCC (India) Limited","marketType":"N","exDate":"06-SEP-18","bcStartDate":"10-SEP-18","css_status_desc":"Listed","dayHigh":"53.55","basePrice":"53.05","securityVar":"10.35","pricebandlower":"47.75","sellQuantity5":"-","sellQuantity4":"-","sellQuantity3":"-","cm_adj_high_dt":"08-DEC-17","sellQuantity2":"-","dayLow":"51.55","sellQuantity1":"40,722","quantityTraded":"71,35,742","pChange":"-2.64","totalTradedValue":"3,714.15","deliveryToTradedQuantity":"40.23","totalBuyQuantity":"-","averagePrice":"52.05","indexVar":"-","cm_ffm":"2,424.24","purpose":"ANNUAL GENERAL MEETING/DIVIDEND RE 0.56 PER SHARE","buyPrice2":"-","secDate":"7DEC2018","buyPrice1":"-","high52":"266.00","previousClose":"53.05","ndEndDate":"-","low52":"50.80","buyPrice4":"-","buyPrice3":"-","recordDate":"-","deliveryQuantity":"28,70,753","buyPrice5":"-","priceBand":"No Band","extremeLossMargin":"5.00","cm_adj_low_dt":"26-OCT-18","varMargin":"10.35","sellPrice1":"51.80","sellPrice2":"-","totalTradedVolume":"71,35,742","sellPrice3":"-","sellPrice4":"-","sellPrice5":"-","change":"-1.40","surv_indicator":"-","ndStartDate":"-","buyQuantity4":"-","isExDateFlag":false,"buyQuantity3":"-","buyQuantity2":"-","buyQuantity1":"-","series":"EQ","faceValue":"1.00","buyQuantity5":"-","closePrice":"51.80","open":"53.15","isinCode":"INE095N01031","lastPrice":"51.65"}],"optLink":"/marketinfo/sym_map/symbolMapping.jsp?symbol=NBCC&amp;instrument=-&amp;date=-&amp;segmentLink=17&amp;symbolCount=2","otherSeries":["EQ"],"futLink":"/live_market/dynaContent/live_watch/get_quote/GetQuoteFO.jsp?underlying=NBCC&amp;instrument=FUTSTK&amp;expiry=27DEC2018&amp;type=-&amp;strike=-","lastUpdateTime":"07-DEC-2018 15:59:59"}</div>'''
html = bs4.BeautifulSoup(r)
soup = html.find('div', {'id':'responseDiv'}).text
data = json.loads(soup)
last_price = data['data'][0]['lastPrice']

编辑:

json_d = soup.find(id='responseDiv')

尝试更改为

json_d = soup.find(‘div’, {‘id’:'responseDiv'})

然后你应该能够做

data = json.loads(json_d)
last_price = data['data'][0]['lastPrice']

看看这是否有帮助。我目前在周二之前都不在电脑上,所以在iPhone上打这个,所以无法测试/玩它。

另一件事是,该网站可能需要在加载后被读入。在这种情况下,我认为您需要查看selenium包或html请求包。

再说一次,我要到星期二才能回家拿笔记本电脑。

最新更新