python使用池进行多重处理

Question

我的Python课程有一项作业。

问题是：

您将编写一个创建多个进程的程序(p=10(。每个流程将收到整数值(y(并计算y乘以4(y*4(。y值的集合是一个列表整数[0，1，2，3，4，5]。你必须使用一个进程池，所以你需要映射这10个处理以获取y整数列表。执行以下操作：

你的程序应该打印出以下三个。(1( 打印出原始列表。(2( 按随机顺序打印每个正在运行的进程名称及其输出y*4执行

(1(的输出：输入列表：[0,1,2,3,4,5]

(2(的输出：按随机顺序输出：

PoolWorker-10输出：[0，4，8，12，16，20]

PoolWorker-11输出：[0，4，8，12，16，20]

PoolWorker-12输出：[0，4，8，12，16，20]

我的第一次尝试是：

import time
from multiprocessing import Pool
from multiprocessing import Process, current_process
def f(number):
result=4*number
Process_name=current_process().name
print(f"Poolworker -{Process_name} output: {result} ")
return result
if __name__=='__main__':
p= Pool(processes=10)
numbers=[0,1,2,3,4,5]
result=p.map(f,numbers)

输出为：

Poolworker -SpawnPoolWorker-2 output: 0 
Poolworker -SpawnPoolWorker-2 output: 4 
Poolworker -SpawnPoolWorker-2 output: 8 
Poolworker -SpawnPoolWorker-2 output: 12 
Poolworker -SpawnPoolWorker-2 output: 16 
Poolworker -SpawnPoolWorker-2 output: 20

我的第二次尝试：

import time
from multiprocessing import Pool
from multiprocessing import Process, current_process
def f(*number):
	numbers=list(number)
	i=0
	for x in numbers:
		numbers[i]=4*numbers[i]
		i+=1
	Process_name=current_process().name
	print(f"Poolworker -{Process_name}output: {numbers} ")
	return numbers
if __name__ == '__main__':
		array=[0,1,2,3,4,5]
		p=Pool(processes=10)
	
		result=p.map(f,array)
		

我仍然有错误的输出与变化。

如何获得所需的输出？

Answer 1

问题是f应用于列表中的每个数字，而您希望每个工作人员在整个列表上进行计算。为了得到这些结果，对代码进行了一些小的修改。

from itertools import repeat
from multiprocessing import Pool
from multiprocessing import Process, current_process
def f(numbers):
result= [4 * num for num in numbers]
name=current_process().name
print(f"Poolworker - {name} output: {result} ")
return result
if __name__=='__main__':
WORKERS = 10
p = Pool(processes=WORKERS)
numbers = [0, 1, 2, 3, 4, 5]
print(f"Original - {numbers}")
result = p.map(f, repeat(numbers, WORKERS))

这将返回以下内容：

Original - [0, 1, 2, 3, 4, 5]
Poolworker - ForkPoolWorker-1 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-2 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-3 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-4 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-5 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-6 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-7 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-8 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-9 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-10 output: [0, 4, 8, 12, 16, 20]

Answer 2

根据您的描述，您希望所有进程将数组乘以4。

因此，使用并行进程，每个进程都有一个要相乘的数字数组。

参考

代码

from multiprocessing import Process
import multiprocessing as mp
def f(numbers, output):
" Places process name and numbers multiplied by 4 into output queue "
output.put((mp.current_process().name, [4*n for n in numbers]))
# Define an output queue
output = mp.Queue()
# Setup a list of processes that we want to run
numbers = [1, 2, 3, 4, 5]
processes = [mp.Process(target=f, args=([1, 2, 3, 4, 5], output)) for x in range(10)]
# Run processes
for p in processes:
p.start()
# Exit the completed processes
for p in processes:
p.join()
# Get process results from the output queue
results = [output.get() for p in processes]

for id, values in results:
print(f'PoolWorker {id} output: {values}')

输出

PoolWorker Process-1 output: [4, 8, 12, 16, 20]
PoolWorker Process-3 output: [4, 8, 12, 16, 20]
PoolWorker Process-9 output: [4, 8, 12, 16, 20]
PoolWorker Process-6 output: [4, 8, 12, 16, 20]
PoolWorker Process-7 output: [4, 8, 12, 16, 20]
PoolWorker Process-5 output: [4, 8, 12, 16, 20]
PoolWorker Process-4 output: [4, 8, 12, 16, 20]
PoolWorker Process-8 output: [4, 8, 12, 16, 20]
PoolWorker Process-2 output: [4, 8, 12, 16, 20]
PoolWorker Process-10 output: [4, 8, 12, 16, 20]