多进程.如何包装** args和**kwargs

Question

我正试图让multiprocess.apply_async接受*args和**kwargs。文档指出，这可以通过调用序列实现:

apply_async(func[, args[, kwds[, callback]]])

但是我不知道如何得到调用语法正确。最简单的例子:

from multiprocessing import Pool
def f(x, *args, **kwargs):
    print x, args, kwargs
args, kw = (), {}
print "# Normal call"
f(0, *args, **kw)
print "# Multicall"
P = Pool()
sol = [P.apply_async(f, (x,), *args, **kw) for x in range(2)]
P.close()
P.join()
for s in sol: s.get()

输出

# Normal call
0 () {}
# Multicall
0 () {}
1 () {}

当args不是空元组时，例如args = (1,2,3)，单调用工作，但多处理解决方案给出:

# Normal call
0 (1, 2, 3) {}
# Multicall
Traceback (most recent call last):
  File "kw.py", line 16, in <module>
    sol = [P.apply_async(f, (x,), *args, **kw) for x in range(2)]
TypeError: apply_async() takes at most 5 arguments (6 given)

使用kwargs参数我得到，例如kw = {'cat':'dog'}

# Normal call
0 () {'cat': 'dog'}
# Multicall
Traceback (most recent call last):
  File "kw.py", line 15, in <module>
    sol = [P.apply_async(f, (x,), *args, **kw) for x in range(2)]
TypeError: apply_async() got an unexpected keyword argument 'cat'

如何正确包装multiprocess.apply_async ?

Answer 1

您不必显式地使用*和**。只需传递元组和字典，并让apply_async解压缩它们:

from multiprocessing import Pool
def f(x, *args, **kwargs):
    print x, args, kwargs
args, kw = (1,2,3), {'cat': 'dog'}
print "# Normal call"
f(0, *args, **kw)
print "# Multicall"
P = Pool()
sol = [P.apply_async(f, (x,) + args, kw) for x in range(2)]
P.close()
P.join()
for s in sol: s.get()

输出:

# Normal call                                                                                        
0 (1, 2, 3) {'cat': 'dog'}
# Multicall
0 (1, 2, 3) {'cat': 'dog'}
1 (1, 2, 3) {'cat': 'dog'}

请记住，在python的文档中，如果一个函数接受*args和**kwargs，它的签名显式地声明:

the_function(a,b,c,d, *args, **kwargs)

在你的例子中:

apply_async(func[, args[, kwds[, callback]]])

没有*，因此args是一个参数，在调用func时解压缩，kwargs是一个参数，处理方式相同。还要注意，在**kwargs:

之后不可能有其他参数。

>>> def test(**kwargs, something=True): pass
  File "<stdin>", line 1
    def test(**kwargs, something=True): pass
                     ^
SyntaxError: invalid syntax