如何在python中的api分页循环后编写可读的csv文件

Question

我在将api调用循环的输出写入可读的csv文件时遇到问题。我对python很陌生，可能是因为数据格式不同。csv只包含9列，它不会将所有其他元素作为单独的列读取

import requests
import json
import pandas as pd
FileNaam = 'name'

params = (
('api_key', 'api_key'),
('since', '1595590000'),
('sort', 'asc')
)
url = 'https://a.klaviyo.com/api/v1/metrics/timeline'
discover_api = requests.get(url, params=params).json()

events = discover_api["data"]
next_id = discover_api["next"]
continueloop = 1
while continueloop ==1:
params = (
('api_key', 'api_key'),
('since', next_id),
('sort', 'asc')
)
discover_apiloop = requests.get(url, params=params).json()
events.extend(discover_apiloop["data"])
if discover_apiloop["next"] is None:
continueloop = 0
else:
next_id = discover_apiloop["next"]

data = json.dumps(events, sort_keys=True, indent=4)
df = pd.read_json(data)
df.to_csv(FileNaam+'.csv')

当Im不循环而只执行一个api调用时，Im使用不同的格式，csv很好。所以下面的代码工作正常，但我希望在使用循环时得到相同的结果。

import requests
import json
import pandas as pd
FileNaam = 'name'
params = (
('api_key', 'api_key'),
('sort', 'asc'),
('since', '1595590000')
)
response = requests.get('https://a.klaviyo.com/api/v1/metrics/timeline', params=params)
responsetext = response.text
data = json.loads(responsetext)
df = pd.json_normalize(data['data'])
df.to_csv(FileNaam +'.csv')

如何转换我的输出以获得正确的csv？

Answer 1

在循环后使用json_normalize使其工作

import requests
import json
import pandas as pd
bestandsNaam = 'name'
params = (
('api_key', 'api_key'),
('since', '1595650000'),
('sort', 'asc')
)
url = 'https://a.klaviyo.com/api/v1/metrics/timeline'
discover_api = requests.get(url, params=params).json()
events = discover_api["data"]
next_id = discover_api["next"]
continueloop = 1
while continueloop ==1:
params = (
('api_key', 'api_key'),
('since', next_id),
('sort', 'asc')
)
discover_apiloop = requests.get('https://a.klaviyo.com/api/v1/metrics/timeline', params=params).json()
events.extend(discover_apiloop["data"])
if discover_apiloop["next"] is None:
continueloop = 0
else:
next_id = discover_apiloop["next"]
df = pd.json_normalize(events)
df.to_csv(bestandsNaam+'.csv')

Answer 2

我正在Klaviyo中尝试分页，数据似乎即将到来，你能告诉我分页是否正确吗？

导入请求导入json进口熊猫作为pdurl＝"；https://a.klaviyo.com/api/v1/metrics/timeline"从日期时间导入时间增量从日期时间导入日期时间，时间

午夜=datetime.component(datetime.today((，time.min(从日期时间导入时间增量

yesterday_midnight=午夜-时间增量(天=1(

yesteday_epoch=str(int((yesterday_midnight(.timestamp((((打印(yested_epoch(querystring＝{"api_key"："api_key"，"since"：yesteday_epoch，"count"："100"，"sort"："desc"}

headers＝{"Accept"："application/json"}

response=requests.request("GET"，url，headers=headers，params=querystring(

output_data_list=response.json(([data']

next_page=response.json((['next']

got_next=真while(got_next(：url＝"；https://a.klaviyo.com/api/v1/metrics/timeline"querystring＝｛"api_key"："pk_api_key"，"since"：next_page，"count"："100"，"sort"："desc"｝

headers = {"Accept": "application/json"}
response = requests.request("GET", url, headers=headers, params=querystring)
output_data_list = response.json()['data']
if(next_page):
with open("pagination1/metrics_{}.json".format(next_page),"a") as f: 
for i in output_data_list:          
f.write(json.dumps(i)+"n")
if next_page is None:
got_next = False
else:
next_page = response.json()['next']
got_next = False