当异步任务正在执行_blocking_工作时，如何在键盘中断时优雅地关闭

Question

更新：asyncio只需按照指示执行即可，您可以很好地处理这些异常-请参阅我的后续答案，我已将其标记为该问题的解决方案。下面是原始问题，并对示例进行了轻微修改，以澄清问题及其解决方案。

我一直在尝试调试一个严重依赖异步的库。在编写一些示例代码时，我意识到执行键盘中断(CTRL-C(有时(很少！(会触发可怕的。。。

Task exception was never retrieved

我一直在努力确保我派生的所有任务都能优雅地处理asyncio.CancelledError，在花了太多时间调试之后，我意识到，如果其中一个异步任务被阻塞操作，我只有才会收到这个错误消息。

阻塞？你真的不应该在任务中执行阻塞工作——这就是为什么asyncio足够善意地警告你这一点。运行以下代码。。。

import asyncio
from time import sleep

async def possibly_dangerous_sleep(i: int, use_blocking_sleep: bool = True):
try:
print(f"Sleep #{i}: Fine to cancel me within the next 2 seconds")
await asyncio.sleep(2)
if use_blocking_sleep:
print(
f"Sleep #{i}: Not fine to cancel me within the next 10 seconds UNLESS someone is"
" awaiting me, e.g. asyncio.gather()"
)
sleep(10)
else:
print(f"Sleep #{i}: Will sleep using asyncio.sleep(), nothing to see here")
await asyncio.sleep(10)
print(f"Sleep #{i}: Fine to cancel me now")
await asyncio.sleep(2)
except asyncio.CancelledError:
print(f"Sleep #{i}: So, I got cancelled...")
raise

def done_cb(task: asyncio.Task):
name = task.get_name()
try:
task.exception()
except asyncio.CancelledError:
print(f"Done: Task {name} was cancelled")
pass
except Exception as e:
print(f"Done: Task {name} didn't handle exception { e }")
else:
print(f"Done: Task {name} is simply done")

async def start_doing_stuff(collect_exceptions_when_gathering: bool = False):
tasks = []
for i in range(1, 7):
task = asyncio.create_task(
possibly_dangerous_sleep(i, use_blocking_sleep=True), name=str(i)
)
task.add_done_callback(done_cb)
tasks.append(task)
# await asyncio.sleep(3600)
results = await asyncio.gather(*tasks, return_exceptions=collect_exceptions_when_gathering)

if __name__ == "__main__":
try:
asyncio.run(start_doing_stuff(collect_exceptions_when_gathering=False), debug=True)
except KeyboardInterrupt:
print("User aborted through keyboard")

调试控制台将告诉您以下内容：

Executing <Task finished name='Task-2' coro=<possibly_dangerous_sleep() done, defined at ~/src/hej.py:5> result=None created at ~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/tasks.py:337> took 10.005 seconds

请放心，上面对sleep(10)的调用并不是我正在处理的库中的罪魁祸首，但它说明了我遇到的问题：如果我试图在上面的测试应用程序运行的前2到12秒内中断它，调试控制台将以一个巨大的源回溯结束：

Fine to cancel me within the next 2 seconds
Not fine to cancel me within the next 10 seconds UNLESS someone is awaiting me, e.g. asyncio.gather()
^CDone with: <Task finished name='Task-2' coro=<possibly_dangerous_sleep() done, defined at ~/src/hej.py:5> exception=KeyboardInterrupt() created at ~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/tasks.py:337>
User aborted through keyboard
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<dangerous_sleep() done, defined at ~/src/hej.py:5> exception=KeyboardInterrupt() created at ~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/tasks.py:337>
source_traceback: Object created at (most recent call last):
File "~/.pyenv/versions/3.10.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "~/.pyenv/versions/3.10.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "~/.vscode/extensions/ms-python.python-2021.12.1559732655/pythonFiles/lib/python/debugpy/__main__.py", line 45, in <module>
cli.main()
File "~/.vscode/extensions/ms-python.python-2021.12.1559732655/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 444, in main
run()
File "~/.vscode/extensions/ms-python.python-2021.12.1559732655/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 285, in run_file
runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
File "~/.pyenv/versions/3.10.0/lib/python3.10/runpy.py", line 269, in run_path
return _run_module_code(code, init_globals, run_name,
File "~/.pyenv/versions/3.10.0/lib/python3.10/runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "~/.pyenv/versions/3.10.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "~/src/hej.py", line 37, in <module>
asyncio.run(start_doing_stuff(), debug=True)
File "~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/base_events.py", line 628, in run_until_complete
self.run_forever()
File "~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/base_events.py", line 595, in run_forever
self._run_once()
File "~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/base_events.py", line 1873, in _run_once
handle._run()
File "~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "~/src/hej.py", line 28, in start_doing_stuff
task = asyncio.create_task(dangerous_sleep())
File "~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/tasks.py", line 337, in create_task
task = loop.create_task(coro)
Traceback (most recent call last):
File "~/src/hej.py", line 37, in <module>
asyncio.run(start_doing_stuff(), debug=True)
File "~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/base_events.py", line 628, in run_until_complete
self.run_forever()
File "~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/base_events.py", line 595, in run_forever
self._run_once()
File "~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/base_events.py", line 1873, in _run_once
handle._run()
File "~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "~/src/hej.py", line 14, in dangerous_sleep
sleep(10)
KeyboardInterrupt

如果我将等待asyncio.sleep(3600)替换为await asyncio.gather(task)(请参阅示例代码(并调用CTRL-C，那么我会在调试控制台中得到一个非常整洁的关闭序列：

Fine to cancel me within the next 2 seconds
Not fine to cancel me within the next 10 seconds UNLESS someone is awaiting me, e.g. asyncio.gather()
^CDone with: <Task finished name='Task-2' coro=<possibly_dangerous_sleep() done, defined at ~/src/hej.py:5> exception=KeyboardInterrupt() created at ~/.pyenv/versions/3.10.0/lib/python3.10/asyncio/tasks.py:337>
User aborted through keyboard

有人能向我解释一下这是不是故意的吗？当asyncio.run((中断时(在清理其自身时(，我本以为所有异步任务都会被取消。

Answer 1

摘要：您需要处理异常，否则asyncio会抱怨。

对于后台任务(即使用gather()没有明确等待的任务(

您可能认为，尝试在任务中使用except asyncio.CancelledError捕获取消(并重新引发它(可以处理所有类型的取消。事实并非如此。如果您的任务在被取消时正在执行阻塞工作，那么您将无法捕获任务本身中的异常(例如KeyboardInterrupt(。这里的安全赌注是在asyncio.Task上使用add_done_callback注册一个已完成的回调。在这个回调中，检查是否存在异常(请参阅问题中更新的示例代码(。如果您的任务在被取消时被阻塞，则done回调将告诉您任务已完成(vs已取消(。

对于使用gather()等待的一堆任务

如果使用聚集，则不需要添加已完成的回调。相反，要求它返回任何异常，它就会很好地处理KeyboardInterrupt。如果不执行此操作，在其任意可用项中引发的第一个异常将立即传播到gather()上等待的任务。如果任务中的KeyboardInterrupt被阻塞，则KeyboardInterrupt将被重新引发，您需要处理它。或者，使用try/except来处理引发的任何异常。请在示例代码中设置collect_exceptions_when_gathering变量，自己尝试一下。

最后：我现在唯一不明白的是，如果用一个任务调用gather()，而不是要求它返回异常，那么我看不到会引发任何异常。尝试修改示例代码使其范围为range(1,2)，这样就不会在CTRL-C上得到混乱的堆栈跟踪。。。？