Python文件抛出json解码错误，如何调整

Question

这段代码一直抛出一个错误，我无法使它运行以保存到文本文件。它总是卡在

Traceback(最近一次调用):文件"c:Python39scrape2.py"，第32行，在响应=请求。Get (url % page, headers=headers).json()第918行，在json
引发RequestsJSONDecodeError(e。Msg, e.doc, e.pos)requests.exceptions.JSONDecodeError: [Errno expected value]: 0

import requests
import json
page = 1
url = f"https://api-prod.grip.events/1/container/4368/search?search=&sort=name&order=asc&type_id=4907,4906,5265,4964,4904,1026,4908&page=%d"
headers = {
'authority': 'api-prod.grip.events',
'accept': 'application/json',
'accept-language': 'en-gb',
'content-type': 'application/json',
'if-none-match': 'W/"7132-A/vrxQVW3GqTDiJFLQqx9lN+Y0s"',
'login-source': 'web',
'origin': 'https://connect.money2020.com',
'referer': 'https://connect.money2020.com/money2020europe/app/home/network/list/34589',
'sec-ch-ua': '" Not A;Brand";v="99", "Chromium";v="101", "Google Chrome";v="101"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"Windows"',
'sec-fetch-dest': 'empty',
'sec-fetch-mode': 'cors',
'sec-fetch-site': 'cross-site',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.67 Safari/537.36',
'x-authorization': 'a422cc2a-31fb-4b4e-a1bd-a34b561adc6c',
'x-grip-version': 'Web/8.3.11',
}

s = requests.Session()
response = requests.post(url, headers=headers)
with open("list.txt", "w") as f:
for page in range(1, 1000):
response = requests.get(url % page, headers=headers).json()
contacts = response("data")
for contact in contacts:
target = "%st%st%st%s" % (contact["company_name"], contact["job_title"], contact["name"],  contact["job_industry"])
f.write(target + "n")
print(target)

Answer 1

服务器返回HTTP代码304(未修改)，因为if-not-match标头已经匹配ETag服务器端(因为这可能是从浏览器devtools复制的)。

只要去掉这个标题(和其他一些不必要的)，并修复拼写错误(contacts = response["data"])。

import requests
import json
url = "https://api-prod.grip.events/1/container/4368/search?search=&sort=name&order=asc&type_id=4907,4906,5265,4964,4904,1026,4908&page=%d"
headers = {
'x-authorization': 'a422cc2a-31fb-4b4e-a1bd-a34b561adc6c'
}
with open("list.txt", "w") as f:
for page in range(1, 1000):
response = requests.get(url % page, headers=headers).json()
contacts = response["data"]
for contact in contacts:
target = "%st%st%st%s" % (contact["company_name"], contact["job_title"], contact["name"],  contact["job_industry"])
f.write(target + "n")
print(target)

您可能还想检查csv模块用于编写TSV文件。

Answer 2

您确定响应是有效的json吗?它可能会收到一个错误，而你还没有处理这种情况。

尝试将其更新为以下内容，它应该会打印出任何错误。

with open("list.txt", "w") as f:
for page in range(1, 1000):
try:
response = requests.get(url % page, headers=headers)
if response.status_code == 200:
response = response.json()
contacts = response("data")
for contact in contacts:
target = "%st%st%st%s" % (contact["company_name"], contact["job_title"], contact["name"],  contact["job_industry"])
f.write(target + "n")
print(target)
else:
print(f"Unsuccessful request: {response}")
except Exception as e:
print(f"Error: {e}")