具有多处理、池和读取文件的 Control-C 处理

Question

我刚刚开始使用Multiprocessing和Python，我需要一些帮助在我的程序中捕获Control-C。 我正在制作的脚本将在文件中读取，然后在每一行上执行一些任务。在任何人评论I/O和多处理的优缺点之前，我知道:)这些任务非常适合多线程。

我有以下代码，从文档中，我希望它可以工作，但是它没有捕获我的键盘异常！ 嘶...... 请帮忙

在 Win10 上运行，如果这有任何区别：

from multiprocessing import cpu_count
from multiprocessing.dummy import Pool as ThreadPool
import argparse
from time import sleep
import signal
import sys

def readfile(file):
with open(file, 'r') as file:
data = file.readlines()
file.close()
return data
def work(line):
while(True):
try:
print(f"rgoing to do some work on {line}")
countdown(5)
except (KeyboardInterrupt, SystemExit):
print("Exiting...")
break
def countdown(time=30):
sleep(time)
def parseArgs(args):
if args.verbose:
verbose = True
print("[+] Verbosity turned on")
else:
verbose = False
if args.threads:
threads = args.threads
else:
threads = cpu_count()
print(f'[+] Using {threads} threads')
return threads, verbose, args.file

if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("-f", "--file", required = True, help="Insert the flie you plan on parsing")
parser.add_argument("-t", "--threads", help="Number of threads, by default will use all available processors")
parser.add_argument("-v", "--verbose", help="increase output verbosity",
action="store_true")
threads, verbose, filename = parseArgs(parser.parse_args())
#read the entire file and store it in a variable:
data = readfile(filename)
#Init the data pool
pool = ThreadPool(threads) # Number of threads going to use
try:
pool.map(work,data) # This launches the workers at the function to do work
except KeyboardInterrupt:
print("Exiting...")
finally:
pool.close()
pool.join()

Answer 1

当您使用 Control-C 时，程序可能正在等待所有线程完成pool.join()。pool.map函数只是启动所有进程，但不会阻塞。这意味着在键盘中断发生时，它不会被捕获，因为程序不在try块内。

我不太确定这里的最佳实践，但我会尝试：

try: 
pool.map(work, data) # This launches the workers at the function to do work 
pool.close() 
pool.join()
except KeyboardInterrupt: 
print("Exiting...")