python的每个运行async函数没有await和parallel

Question

我在我的CSV中有10个链接，我试图在getTasks函数的循环中同时运行所有链接。然而，它现在的工作方式是，它向链接1发送请求，等待它完成，然后是链接2，等等。我想要10个链接，我必须运行所有每当startTask被调用，导致每秒10个请求。

有人知道如何使用下面的代码编码吗?提前谢谢。

import requests
from bs4 import BeautifulSoup
import asyncio
def getTasks(tasks):
for task in tasks:
asyncio.run(startTask(task))

async def startTask(task):

success = await getProduct(task)
if success is None:
return startTask(task)
success = await addToCart(task)
if success is None:
return startTask(task)
...
...
...
getTasks(tasks)

Answer 1

首先，为了使请求并发发送，您应该使用aiohttp而不是阻塞I/O的requests包。并使用asyncio的信号量来限制同时并发进程的数量。

import asyncio
import aiohttp
# read links from CSV
links = [
...
]
semaphore = asyncio.BoundedSemaphore(10) 
# 10 is the max count of concurrent tasks
# that can be processed at the same time.
# In this case, tasks are requests.
async def async_request(url):
async with aiohttp.ClientSession() as session:
async with semaphore, session.get(url) as response:
return await response.text()

async def main():
result = await asyncio.gather(*[
async_request(link) for link in links
])
print(result)  # [response1, response2, ...]

if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()