Web使用API从交易所抓取数据

Question

我正在寻找网页抓取包含"内幕股票交易数量"的第二张表。来自以下网站:

https://www.nasdaq.com/market-activity/stocks/aapl/insider-activity

如果可能的话，我最好需要有人来展示如何使用纳斯达克api。我相信我通常使用的webscraper(使用beautifulSoup)对于这个任务是低效的。

我有一些现有的代码，帮助获取数据从同一网站使用它的api，但不同的信息。最好，我只需要一个不同的api端点，然后按照类似于下面代码的结构进行一些调整:

import requests
import json
nasdaq_dict = {}
url = 'https://api.nasdaq.com/api/company/AAPL/institutional-holdings?limit=15&type=TOTAL&sortColumn=marketValue&sortOrder=DESC'
headers = {
'accept': 'application/json, text/plain, */*',
'origin': 'https://www.nasdaq.com',
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.79 Safari/537.36'
}
r = requests.get(url, headers=headers)
nasdaq_dict['activePositions'] = r.json()['data']['activePositions']['rows']
nasdaq_dict['newSoldOutPositions'] = r.json()['data']['newSoldOutPositions']['rows']
with open('AAPL_institutional_holdings.json', 'w') as f:
json.dump(nasdaq_dict, f, indent=4)

Answer 1

这里有一种获取数据的方法(作为字典:如果您希望它作为表，请说明):

import requests
headers = {
'accept-language': 'en-US,en;q=0.9',
'origin': 'https://www.nasdaq.com/',
'referer': 'https://www.nasdaq.com/',
'accept': 'application/json, text/plain, */*',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36'
}
data = requests.get('https://api.nasdaq.com/api/company/AAPL/insider-trades?limit=15&type=ALL&sortColumn=lastDate&sortOrder=DESC', headers=headers).json()['data']['numberOfSharesTraded']
print(data)

终端结果:

{'headers': {'insiderTrade': 'INSIDER TRADE', 'months3': '3 MONTHS', 'months12': '12 MONTHS'}, 'rows': [{'insiderTrade': 'Number of Shares Bought', 'months3': '0', 'months12': '0'}, {'insiderTrade': 'Number of Shares Sold', 'months3': '1,317,881', 'months12': '1,986,819'}, {'insiderTrade': 'Total Shares Traded', 'months3': '1,317,881', 'months12': '1,986,819'}, {'insiderTrade': 'Net Activity', 'months3': '(1,317,881)', 'months12': '(1,986,819)'}]}