我正试图在类中使用python 3中的multiprocessing.pool。这是最初的功能：
stress_matrix, compliance = self.problem.compute_objs(self.xphys)
avg_sm = np.zeros(self.nel)
for i2 in range(self.nel):
avg_sm = avg_stress_calc(self.xphys, self.nelx, self.nely,
self.rmin, stress_matrix, avg_sm, i2)
这给我留下了一个形状为(16(的数组，是这样的吗：
array([0.81814754, 0.64561319, 0.62517261, 0.78422925, 0.6962134 ,
0.65993462, 0.63970099, 0.68776093, 0.49890513, 0.60900864,
0.71575952, 0.73120825, 0.32964378, 0.53196899, 0.80481781,
0.99930964])
我试图通过使用多处理池来加快速度，因为我的NEL大于10000(正常情况下(，如下所示：
avgsm = np.zeros(self.nel)
pool = multiprocessing.Pool()
func = partial(avg_stress_calc, self.xphys, self.nelx,
self.nely, self.rmin, stress_matrix, avgsm)
avg_sm = pool.map(func, range(self.nel))
出于某种原因，当我这样做时，我会得到一个属性错误："list"对象没有属性"shape"，我将其转换为数组以获得它的形状(16，16(。多进程版本的输出如下：
[array([0.81814754, 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. ]), array([0. , 0.64561319, 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. ]), array([0. , 0. , 0.62517261, 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. ]), array([0. , 0. , 0. , 0.78422925, 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. ]), array([0. , 0. , 0. , 0. , 0.6962134, 0. ,
0. , 0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. ]), array([0. , 0. , 0. , 0. , 0. ,
0.65993462, 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. ]), array([0. , 0. , 0. , 0. , 0. ,
0. , 0.63970099, 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. ]), array([0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0.68776093, 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. ]), array([0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0.49890513, 0. ,
0. , 0. , 0. , 0. , 0. ,
0. ]), array([0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0.60900864,
0. , 0. , 0. , 0. , 0. ,
0. ]), array([0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0.71575952, 0. , 0. , 0. , 0. ,
0. ]), array([0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0.73120825, 0. , 0. , 0. ,
0. ]), array([0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0.32964378, 0. , 0. ,
0. ]), array([0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0.53196899, 0. ,
0. ]), array([0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0.80481781,
0. ]), array([0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0.99930964])]
我希望使用多处理来加速for循环，因为这是我代码中最耗时的部分。如果能就我做错了什么提出任何建议，我们将不胜感激。谢谢
Pool().map()是不错的，但如果您需要在不同实例之间共享对象(内存空间(，则最好使用这样的线程：
import numpy as np
from threading import Thread
import random
import time
def partial(args, process_id , avg_sm):
# do some stuff
print("Th%d - Starting working time from 1 to 3 seconds"%process_id)
# do more stuff
working_time = random.uniform(1,3)
time.sleep( working_time )
# write the result
avg_sm[process_id] = random.uniform(0.0,1.0)
print("Th%d - Finishing working time from 1 to 3 seconds"%process_id)
if __name__ == "__main__":
MAX_THREADS = 1000
nel = 10000
args = ["some args you define"]
avg_sm = np.zeros(nel) #[0]*nel
process_l = []
t0 = time.time()
for pid in range(nel):
if pid%MAX_THREADS == 0:
for p in process_l:
p.join()
process_l.append( Thread( target=partial, args=( args, pid, avg_sm ) ) )
process_l[pid].start()
for p in process_l:
p.join()
t1 = time.time() - t0
print(avg_sm)
print("Total computation time: %d s"%t1)
PS：你可以放任意多的args，在这个例子中，我使用了一个名为args的列表
编辑
由于内存问题，需要MAX_THREADS限制，您必须自己定义。当创建的线程达到这个数量(if pid%MAX_THREADS == 0:(时，您将等待它们完成(也许您可以在一个或多个线程完成后创建一个新线程，我认为这会更有效率(。我假设您的方法延迟1到3秒来完成计算时间。
结果：
Th9889 - Starting working time from 1 to 3 seconds
Th9988 - Starting working time from 1 to 3 seconds
Th9919 - Starting working time from 1 to 3 seconds
Th9986 - Starting working time from 1 to 3 seconds
Th9866 - Starting working time from 1 to 3 seconds
Th9951 - Starting working time from 1 to 3 seconds
Th9918 - Starting working time from 1 to 3 seconds
Th9991 - Starting working time from 1 to 3 seconds
Th9886 - Starting working time from 1 to 3 seconds
Th9915 - Starting working time from 1 to 3 seconds
Th9996 - Starting working time from 1 to 3 seconds
Th9963 - Starting working time from 1 to 3 seconds
Th9978 - Starting working time from 1 to 3 seconds
[0.01340808 0.0567745 0.31191508 ... 0.91127015 0.95141791 0.60075809]
Total computation time: 42 s