尽管设置了工人参数，但 Ssipy 的differential_evolution无法在多个内核中运行

Question

我想用workers参数在多核scipy的differential_evolution函数中运行。

我遵循了文档中的说明，并开发了以下脚本来检查在使用和不使用workers参数的情况下函数的运行时间：

import time
if __name__ == '__main__':
from scipy.optimize import differential_evolution, rosen
bounds = [( 0, 2 ), ( 0, 2 ), ( 0, 2 ), ( 0, 2 ), ( 0, 2 )]
print( 'Beginning optimisation without parallelisation' )
start = time.time()
result = differential_evolution( rosen, bounds,
workers=1,
popsize=200,
maxiter=1000,
tol=0.001,
seed=1 )
end = time.time()
print( 'Optimisation finished after {} seconds'.format( round( end - start, 2 ) ) )
print( 'nBeginning optimisation with workers parameter set to 10' )
start = time.time()
result = differential_evolution( rosen, bounds,
updating='deferred',
workers=10,
popsize=200,
maxiter=1000,
tol=0.001,
seed=1 )
end = time.time()
print( 'Optimisation finished after {} seconds'.format( round( end - start, 2 ) ) )

运行脚本后，以下是每次调用返回的运行时间：

Beginning optimisation without parallelisation
Optimisation finished after 59.0 seconds
Beginning optimisation with workers parameter set to 10
Optimisation finished after 60.43 seconds

正如您所看到的，workers参数设置为10的执行速度甚至比第一个还要慢。你能帮我做这个吗？为了在differential_evolution函数中实现并行化，我还需要做什么吗？

我目前使用的是Python 3.8，我的scipy版本是1.8.1。

Answer 1

@9769953是正确的。multiprocessing引入了一些开销：创建进程、将计算分配给子进程等。只有当计算是CPU密集型的时，才能克服这些开销，而rosen不是。

如果你患有scipy>1.9，如果你的目标函数可以矢量化，那么矢量化可以提供速度增益。如果您可以在矢量化中消除整个计算中常见的中间计算，那么增益将更大：

import time
import numpy as np
from scipy.optimize import differential_evolution, rosen
bounds = [( 0, 2 ), ( 0, 2 ), ( 0, 2 ), ( 0, 2 ), ( 0, 2 )]

print( 'Beginning optimisation without parallelisation' )
start = time.time()
result = differential_evolution( rosen, bounds,
workers=1,
popsize=200,
updating='deferred',
seed=1 )
end = time.time()
print( f'Optimisation finished after {end - start} seconds')

print( 'nBeginning optimisation with vectorisation' )
start = time.time()
result = differential_evolution( rosen, bounds,
updating='deferred',
popsize=200,
seed=1,
vectorized=True
)
end = time.time()
print( f'Optimisation finished after {end-start} seconds')

给出：

Beginning optimisation without parallelisation
Optimisation finished after 46.94838500022888 seconds
Beginning optimisation with vectorisation
Optimisation finished after 40.34918713569641 seconds