python追加在使用多处理时不起作用

Question

问题在于附加"e"到我的名单"ok = []"没有效果，但考虑到这一点很奇怪，当执行print(e)时，只在ok.append(e)上方一行，"e"被打印出来

不需要理解程序和它的作用，这里的主要问题是，向我的列表添加一些值没有效果，即使这个值是真实的。

我试图在if __name__=='__main__':中使用ok =[]，但是这给了我NameError: name 'ok' is not defined错误，所以我然后尝试使用"全局ok"some_function"然而，这给了我相同的结果

import time
import multiprocessing as mp
ratios1 = [1/x for x in range(1,11)]
ratios2 = [y/1 for y in range(1,11)]
x = 283
y = 436
ok = []
def some_function(x_, y_):
list_ = [[a, b] for a in range(1, 1980 + 1) for b in range(1, 1980 + 1) if a / b == x_ / y_]
for e in list_:
if not e[0] in [h[0] for h in ok]:
if not e[1] in [u[1] for u in ok]:
print(e)
ok.append(e)

if __name__=='__main__':
processes = []
if x / y in ratios1 or x / y in ratios2:
some_function(x_=x, y_=y)
else:
for X_, Y_ in [
[x, y],
[x - 1, y], [x, y - 1], [x + 1, y], [x, y + 1],
[x - 2, y], [x, y - 2], [x + 2, y], [x, y + 2],
[x - 3, y], [x, y - 3], [x + 3, y], [x, y + 3]
]:
p = mp.Process(target=some_function, args=(X_,Y_))
processes.append(p)
start = time.time()
for p_ in processes:
p_.start()
for p_ in processes:
p_.join()
end = time.time()
print(f"finished in {end - start} sec")
print(ok)

运行时输出:

[...]                                   # other values of "e"
[283, 433]                              # some random "e" value
[566, 866]                              # some random "e" value
[849, 1299]                             # some random "e" value
[1132, 1732]                            # some random "e" value
finished in 0.8476874828338623 sec      # execution time
[]                                      # the "ok" list being printed out at the end

在"some_function"中添加print(id(ok))最后，它给出了以下输出:

OBS:我删除了输出

的打印(e)

2489040444480
3014871358528
2324227431488
2471301880896
1803966487616
2531583073344
1665411652672
2149818113088
2330038901824
1283883998272
2498472320064
2147028311104
2509405887552
finished in 0.8341867923736572 sec
2589544128640
[]

Answer 1

您需要一个可以从多个进程访问的列表，这是通过使用multiprocessing.Manager.list来实现的，并且您必须将其作为参数传递，您不能将其作为全局变量，因为继承全局变量是特定于操作系统的。

使用托管列表比普通列表慢，所以如果你发现性能不可接受，你应该尝试只使用局部变量，忘记使用全局变量，因为IPC是一个昂贵的过程。

import time
import multiprocessing as mp
ratios1 = [1/x for x in range(1,11)]
ratios2 = [y/1 for y in range(1,11)]
x = 283
y = 436
def some_function(x_, y_, ok_list):
list_ = [[a, b] for a in range(1, 1980 + 1) for b in range(1, 1980 + 1) if a / b == x_ / y_]
for e in list_:
if not e[0] in [h[0] for h in ok_list]:
if not e[1] in [u[1] for u in ok_list]:
print(e)
ok_list.append(e)

if __name__=='__main__':
manager = mp.Manager()
ok_list = manager.list()
processes = []
if x / y in ratios1 or x / y in ratios2:
some_function(x_=x, y_=y)
else:
for X_, Y_ in [
[x, y],
[x - 1, y], [x, y - 1], [x + 1, y], [x, y + 1],
[x - 2, y], [x, y - 2], [x + 2, y], [x, y + 2],
[x - 3, y], [x, y - 3], [x + 3, y], [x, y + 3]
]:
p = mp.Process(target=some_function, args=(X_,Y_,ok_list))
processes.append(p)
start = time.time()
for p_ in processes:
p_.start()
for p_ in processes:
p_.join()
end = time.time()
print(f"finished in {end - start} sec")
print(ok_list)

Answer 2

这应该可以工作，问题是当您启动进程时，它使用的对象并没有真正传递给它，而是克隆了它们。使用muliprocessing.Pool.starmap允许我们从规避此问题的进程返回值。我们使用starmap而不仅仅是map，因此我们可以向some_function传递多个参数。此外，Pool允许您替换for X_,Y_ in ...循环并以多进程方式运行它。

import time
import multiprocessing as mp
from multiprocessing import Pool
ratios1 = [1/x for x in range(1,11)]
ratios2 = [y/1 for y in range(1,11)]
x = 283
y = 436
ok = []
def some_function(x_, y_):
list_ = [[a, b] for a in range(1, 1980 + 1) for b in range(1, 1980 + 1) if a / b == x_ / y_]
for e in list_:
if not e[0] in [h[0] for h in ok]:
if not e[1] in [u[1] for u in ok]:
print(e)
ok.append(e)
return ok
if __name__=='__main__':
processes = []
res=[]
if x / y in ratios1 or x / y in ratios2:
some_function(x_=x, y_=y)
else:
start = time.time()
with Pool(13) as p:
res = p.starmap(some_function, [[x, y],
[x - 1, y], [x, y - 1], [x + 1, y], [x, y + 1],
[x - 2, y], [x, y - 2], [x + 2, y], [x, y + 2],
[x - 3, y], [x, y - 3], [x + 3, y], [x, y + 3]])
ok = res
end = time.time()
print(f"finished in {end - start} sec")
print(ok)