我在多处理器机器上的Linux/Debian测试中使用Python 3.9.2。我正在努力理解多处理是如何工作的。
我写了两个简单的脚本，执行两个指数函数，一个没有多处理，另一个有。
这是一个没有多处理的：
from timeit import default_timer as timer
def sqr(n):
a = n ** n
return a
def sqr_2(m):
b = m ** m
return b
def main():
start = timer()
print(f'sqr = {sqr(100000)}nsqr_2= {sqr_2(200000)}')
end = timer()
print(f'time frame in which the operation is resolved: {end - start} seconds')
if __name__ == '__main__':
main()
这是使用多处理的脚本：
from multiprocessing import Pool, cpu_count
from timeit import default_timer as timer
def sqr_1(n):
return n ** n
def sqr_2(m):
return m ** m
def main():
cpu_cnt = cpu_count()
pool = Pool(processes = cpu_cnt) #In this case there are 12 processors
start = timer()
val_1 = (100000,)
val_2 = (200000,)
process_1 = pool.map_async(sqr_1, val_1)
process_2 = pool.map_async(sqr_2, val_2)
print(f'Results: {process_1.get(), process_2.get()}')
end = timer()
print(f'time frame in which the operation is resolved: {end - start} seconds')
if __name__ == '__main__':
main()
问题是，在没有任何错误的情况下完成的第二个脚本的进程在相同的时间(大约14秒(内执行了与第一个脚本相同的任务。因此，第二个脚本中的多处理不起作用。我提前感谢任何想指出这是错误的人！
请考虑以下脚本。它允许您在运行时选择调用函数的次数，以及是串行调用还是并行调用。它也只是计算价值；它不试图将字符串表示写入标准输出(因为将CCD_ 1的结果转换为字符串对于大的CCD_。
from multiprocessing import Pool, cpu_count
from timeit import default_timer as timer
import sys
def f(n):
return n ** n
def main():
cpu_cnt = cpu_count()
n = int(sys.argv[2])
start = timer()
if sys.argv[1] == "s":
s = [f(100000) for _ in range(n)]
else:
pool = Pool(processes = cpu_cnt)
s = [pool.map_async(f, (100000,)) for _ in range(n)]
results = [x.get() for x in s]
end = timer()
print(f'time frame in which the operation is resolved: {end - start} seconds')
if __name__ == '__main__':
main()
以下是我的4核机器上2、6、12、24、48、96和192个函数调用的结果：
% for n in 2 6 12 24 48 96 192; do print $n; for x in s p; do python3 tmp.py $x $n; done; done
2
time frame in which the operation is resolved: 0.146144435 seconds
time frame in which the operation is resolved: 0.178840965 seconds
6
time frame in which the operation is resolved: 0.423103791 seconds
time frame in which the operation is resolved: 0.24940852500000002 seconds
12
time frame in which the operation is resolved: 0.848754817 seconds
time frame in which the operation is resolved: 0.340022419 seconds
24
time frame in which the operation is resolved: 1.691312521 seconds
time frame in which the operation is resolved: 0.571664972 seconds
48
time frame in which the operation is resolved: 3.415401498 seconds
time frame in which the operation is resolved: 1.029526396 seconds
96
time frame in which the operation is resolved: 6.76773454 seconds
time frame in which the operation is resolved: 2.016387216 seconds
192
time frame in which the operation is resolved: 13.529949021999998 seconds
time frame in which the operation is resolved: 3.770171452 seconds
由于并行化本身的开销，只有两个并行进程，所以没有加速。(事实上，有一个减速。(一旦你开始运行更多的进程，加速就会增加，尽管对于n内核，你永远不会看到n的加速。