多处理器代码在导入时工作,在被调用时中断



在一个名为test.py的文件中,我有

print 'i am cow'
import multi4
print 'i am cowboy'

multi4.py中我有

import multiprocessing as mp
manager = mp.Manager()
print manager

我对这个代码的操作方式感到困惑。

在命令行,如果我键入python,然后在python环境中,如果我输入import test.py,我会得到预期的行为:

Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
>>>import test
i am cow
<multiprocessing.managers.SyncManager object at 0x025209B0>
i am cowboy 
>>>

但是,如果我在命令行中键入test.py,我会得到

i am cow
i am cow
i am cow
i am cow
i am cow
i am cow
i am cow
i am cow
i am cow
i am cow
i am cow
i am cow
i am cow
i am cow
i am cow
i am cow
i am cow
i am cow

除非我杀死它,否则这种情况可能会一直持续下去。当我杀死它时,我会得到一堆重复的错误:

KeyboardInterrupt
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:Python27libmultiprocessingforking.py", line 373, in main
prepare(preparation_data)
File "C:Python27libmultiprocessingforking.py", line 488, in prepare
'__parents_main__', file, path_name, etc
KeyboardInterrupt
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:Python27libmultiprocessingforking.py", line 373, in main
prepare(preparation_data)
File "C:Python27libmultiprocessingforking.py", line 488, in prepare
'__parents_main__', file, path_name, etc
KeyboardInterrupt

那到底发生了什么?为什么它在导入时表现为这样,而在我尝试运行它时表现为另一种?

multiprocessing在Windows上的交互式提示中无法正常工作,因为它无法在其生成的子进程中正确地重新导入__main__。然而,实际上在这里对有帮助,因为它防止manager = mp.Manager()行在Manager启动时产生的子进程中递归执行。

然而,在实际的脚本中,子级可以正确地重新导入__main__。您看到的是无限递归,因为您没有用if __name__ == "__main__":保护对mp.Manager()的调用,这是Windows上防止mp.Manager()在重新导入时在子级中执行所必需的:

import multiprocessing as mp
if __name__ == "__main__":
manager = mp.Manager()
print manager

编辑:

您的示例中,主脚本(test.py)导入创建Manager的模块,需要进行一些重构。您需要通过调用模块中实际使用multiprocessing:的方法,从主脚本实例化Manager

print 'i am cow'
import multi4
if __name__ == "__main__":
multi4.init_manager()
print 'i am cowboy'

multi4.py

import multiprocessing as mp
manager = None
def init_manager():
global manager
manager = mp.Manager()

这是确保仅在实际执行脚本时创建Manager的唯一方法。

最新更新