在尚未启动的池辅助角色中捕获异常

Question

我正在尝试在子进程中捕获异常。如果工作人员数量 == 任务数量，我的代码(见下文(工作正常。但是，如果工人超过示例中的任务，则尚未启动的 2 个进程将出现相同的异常，但我在 MainProcess 和子流程中都无法捕获它。

from multiprocessing import Pool, current_process
import time
from exceptions import GracefulExit
import signal

def terminate_handler(signum, frame):
print(dir(frame))
if current_process().name == 'MainProcess':
# we will not raise error if process
# is main because we need to finish all the jobs
return
raise GracefulExit()

def test_func(val):
i = 0
try:
while True:
time.sleep(0.1)
i += 1
if i >= 10:
return i
except GracefulExit:
pass
return i

if __name__ == "__main__":
signal.signal(
signal.SIGINT, terminate_handler)
try:
with Pool(5) as p:
r = p.map(test_func, [1, 2, 3])
except GracefulExit:
pass
print(r)

Answer 1

我能建议什么，选择任何变体：

1. 确保任务数量不少于工作线程数量，只需添加存根(例如None(确保所有工人都能启动

2. 覆盖池进程run方法：

   class CustomProcess(Process):
   def run(self):
   try:
   super().run()
   except GracefulExit:
   print('interrupted idle worker')
   
   class CustomPool(multiprocessing.pool.Pool):
   Process = CustomProcess
   ....
   with CustomPool(5) as p:
   r = p.map(test_func, [1, 2, 3])
   

3. 编写自己的池实现