加快 python API 请求的速度

Question

嗨，我正在尝试将输入从csv文件传递到我的url，并且文件包含数百万条记录，因此响应非常慢，需要很多时间，有时超时，任何人都可以帮助我使其更快，我可以提供代码，但不能提供数据和URL，因为它是机密的。 这是我的代码：

import pandas as pd
import requests
import json
from pandas.io.json import json_normalize
from flatten_json import flatten
df=pd.read_excel('C:/data1.xlsx',index=None,index_col=None,encoding='UTF-8')
df.head(5)
df.shape
df=df[['NAME','country']]
df.head(5)
passwd=b"abc"
user=b"xxxxx"
# Make a request to the endpoint using the correct auth values
auth_values = (user,passwd)
dd=df.values
dfj = pd.DataFrame()
for i,j in dd:
url='http:xyz.com/&name='+str(i)+'&country='+str(j)    
resp = requests.get(url,auth=auth_values)
r=resp.json()

请修改此代码以使其更快

提前感谢您的帮助

enter code here

Answer 1

试试这个线程。

from concurrent.futures import ThreadPoolExecutor
executors = ThreadPoolExecutor(max_workers = n) # n = int number of threads.
running_threads = []
#To do a job
run_threads = executors.submit(func,foo) #func is your function
running_threads.append(run_threads)
#To wait for your thread to end:
while True: #All inserts are complete
if all(i.done() == True for i in running_threads):
print("all done")
break
else:
time.sleep(1)