Python 2.7:如何弥补缺少的pool.starmap?

Question

>我已经定义了这个函数

def writeonfiles(a,seed):
random.seed(seed)
f = open(a, "w+")
for i in range(0,10):
j = random.randint(0,10)
#print j
f.write(j)
f.close()

其中 a 是包含文件路径的字符串，种子是整数种子。 我想并行化一个简单的程序，使每个内核都采用我放弃的可用路径之一，播种其随机生成器并在该文件上写入一些随机数，因此，例如，如果我传递 向量

vector = [Test/file1.txt, Test/file2.txt] 

和种子

seeds = (123412, 989898), 

它为第一个可用内核提供功能

writeonfiles(Test/file1.txt, 123412) 

第二个是具有不同参数的相同函数：

writeonfiles(Test/file2.txt, 989898)

我已经在Stackoverflow上查看了很多类似的问题，但我无法使任何解决方案起作用。 我尝试的是：

def writeonfiles_unpack(args):
return writeonfiles(*args)
if __name__ == "__main__":
folder = ["Test/%d.csv" %i for i in range(0,4)]
seed = [234124, 663123, 12345 ,123833]
p = multiprocessing.Pool()
p.map(writeonfiles, (folder,seed))

并给了我类型错误：writeonfiles(( 正好需要 2 个参数(给定 1 个(。

我也试过

if __name__ == "__main__":
folder = ["Test/%d.csv" %i for i in range(0,4)]
seed = [234124, 663123, 12345 ,123833]
p = multiprocessing.Process(target=writeonfiles, args= [folder,seed])
p.start()

但它给了我
文件 "/usr/lib/python2.7/random.py"，第 120 行，种子 super(Random， self(.seed(a( 类型错误：不可哈希类型："列表">

最后，我尝试了上下文管理器

@contextmanager
def poolcontext(*args, **kwargs):
pool = multiprocessing.Pool(*args, **kwargs)
yield pool
pool.terminate()
if __name__ == "__main__":
folder = ["Test/%d" %i for i in range(0,4)]
seed = [234124, 663123, 12345 ,123833]
a = zip(folder, seed)
with poolcontext(processes = 3) as pool:
results = pool.map(writeonfiles_unpack,a )

它导致 文件 "/usr/lib/python2.7/multiprocessing/pool.py"，第 572 行，在 get 中 提高self._value

类型错误："模块"对象不可调用

Answer 1

Python2.7 缺少 Python 3.3+ 中的starmap池方法。你可以通过使用包装器装饰目标函数来克服这个问题，包装器解压缩参数元组并调用目标函数：

import os
from multiprocessing import Pool
import random
from functools import wraps

def unpack(func):
@wraps(func)
def wrapper(arg_tuple):
return func(*arg_tuple)
return wrapper
@unpack
def write_on_files(a, seed):
random.seed(seed)
print("%d opening file %s" % (os.getpid(), a))  # simulate
for _ in range(10):
j = random.randint(0, 10)
print("%d writing %d to file %s" % (os.getpid(), j, a))  # simulate

if __name__ == '__main__':
folder = ["Test/%d.csv" % i for i in range(0, 4)]
seed = [234124, 663123, 12345, 123833]
arguments = zip(folder, seed)
pool = Pool(4)
pool.map(write_on_files, iterable=arguments)
pool.close()
pool.join()