使用python/loop提取分页的json数据

Question

我用api提取了json数据。响应示例如下。

{
  "page": 1,
  "pages": 5,
  "limit": 100,
  "count": 466,
  "records": [
    {
      "epoch": "9b4WLoXXYgga9mGRLGsWKu",
      "last_height": 5402,
      "last_time": "2021-02-22T01:02:43.011692Z",
      "expected_blocks": 432,
      "produced_blocks": 432,
    }
  ]
}

但页面限制为100行。参数查询可用于页面。下面是我使用的python代码：

import requests
import json
import pandas as pd
response = requests.get(url, params = {'page': 1})
json_data = json.loads(response.text)
df = pd.DataFrame(json_data['records']) 
print(df)

如何使用循环获取所有页面？

感谢

Answer 1

您可以尝试这样做，并通过循环获得数据帧中的所有数据。

import requests
import json
import pandas as pd
response = requests.get(url, params = {'page': 1}) #---> remember to keep your url 
records = []
for page_number in range(2, response.json().get("pages")+1):
    records += response.json().get('records')    
    response = requests.get(url, params = {'page': page_number})
df = pd.DataFrame(records) 
print(df)