Multiprocessing使用Python 2.7抛出属性错误

Question

from itertools import product
from multiprocessing import Pool
with Pool(4) as pool:
pool.map(lambda x: run_test_function(x, arg2, arg3, arg4), arg1)

执行上述代码后，我得到下面的错误。还有一些其他的代码，我不能在这里写。但是实际的问题只来自于这段代码。

Traceback (most recent call last):
File "modProfileChange_test.py", line 347, in <module>
main(sys.argv[1:])
File "modProfileChange_test.py", line 336, in main
test_run_code(arg1, arg2, arg3, arg4, arg5, arg6)
File "modProfileChange_test.py", line 23, in test_run_code
with Pool(4) as pool:
AttributeError: __exit__

Answer 1

1. 在Python 2.7中，multiprocessing.Pool不是上下文管理器，因此它不能在with语句中使用
   1. 解决方案-使用对变量的常规赋值创建一个池:

      my_pool = Pool(4)
      my_pool.map(...)
      

2. lambda函数不能与multiprocessing.Pool一起工作，即使在Python 3中也是如此。
   1. 解决方案-使用上面链接中的解决方案模拟闭包:

      from functors import partial
      def run_test_function(x, fun_arg2, fun_arg3, fun_arg4):
      # your code here
      ...
      process_func = partial(run_test_function, fun_arg2=arg2, fun_arg3=arg3, fun_arg4=arg4)
      

把这些放在一起:

from multiprocessing import Pool
from functools import partial
def run_test_function(x, fun_arg2, fun_arg3, fun_arg4):
# this is an example
print x, fun_arg2, fun_arg3, fun_arg4
if __name__ == "__main__":
arg1 = 1,2,3,4
arg2 = "hello"
arg3 = "world"
arg4 = "!"
process_func = partial(run_test_function, fun_arg2=arg2, fun_arg3=arg3, fun_arg4=arg4)
my_pool = Pool(4)
my_pool.map(process_func, arg1)

输出:

~/test $ python2.7 so10.py
1 hello world !
2 hello world !
3 hello world !
4 hello world !