如何显示每页超过100个结果

Question

我想把这个页面上的结果数：https://fifatracker.net/players/改为100多个，然后把表导出到Excel，让它对我来说更容易。我试着按照教程使用python来抓取它，但我无法使它工作。如果有一种方法可以从所有页面中提取表格，它也会对我有所帮助

Answer 1

如前所述，每个请求限制为100。只需在api上迭代查询负载即可获得每个页面：

import pandas as pd
import requests
url = 'https://fifatracker.net/api/v1/players/'
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36'}
page= 1 
payload = {
"pagination":{
"per_page":"100","page":page},
"filters":{
"attackingworkrate":[],
"defensiveworkrate":[],
"primarypositions":[],
"otherpositions":[],
"nationality":[],
"order_by":"-overallrating"},
"context":{
"username":"guest",
"slot":"1","season":1},
"currency":"eur"}
jsonData = requests.post(url, headers=headers, json=payload).json()
current_page = jsonData['pagination']['current_page']
last_page = jsonData['pagination']['last_page']
dfs = []
for page in range(1,last_page+1):
if page == 1:
pass

else:
payload['pagination']['page'] = page
jsonData = requests.post(url, headers=headers, json=payload).json()

players = pd.json_normalize(jsonData['result'])
dfs.append(players)
print('Page %s of %s' %(page,last_page))
df = pd.concat(dfs).reset_index(drop=True)

输出：

print(df)
slug  ... info.contract.loanedto_clubname
0            lionel-messi  ...                             NaN
1       cristiano-ronaldo  ...                             NaN
2      robert-lewandowski  ...                             NaN
3               neymar-jr  ...                             NaN
4         kevin-de-bruyne  ...                             NaN
...  ...                             ...
19137           levi-kaye  ...                             NaN
19138      phillip-cancar  ...                             NaN
19139         julio-pérez  ...                             NaN
19140     alan-mclaughlin  ...                             NaN
19141   tatsuki-yoshitomi  ...                             NaN
[19142 rows x 92 columns]