如何在使用多处理器的情况下重新添加字典中的数字

Question

我想构建一个使用多处理的程序。总体任务是对从1到17的数字求和。使用某种"；"水平">

以下是一些例子：

55       <- level 4
36        19  <- level 3
10      26    19  <- level 2
3   7   11  15  19  <- level 1
1 2 3 4 5 6 7 8 9 10 <- level 0

正如你所看到的，我想把两个相邻的数字相加——这是绝对必要的，必须用多处理器处理。我已经达到1级了。现在，我不知道我该如何进行，这里有一个问题要问你——既然我的流程已经完成，我如何才能在各个层面上循环更多？这是我的一些代码：

我有字典

def divide_test(dict):
index = 0
print(len(starting_list))
for i in range(1, len(starting_list), 2):
temporary = []
temporary.append(starting_list[i - 1])
temporary.append(starting_list[i])
print(f"INDEX = {i}, S1 {starting_list[i - 1]}, S2 {starting_list[i]}")
dict[index] = temporary
index += 1
# Last index couldn't be added to rest of the numbers, so I have to save it for later usage.
if None in dict.values():
last_key = next(reversed(dict.keys()))
dict[last_key] = starting_list[-1]
print("nnn")
for key, value in dict.items():
print(f"KEY {key}, VALUE {value}")
return dict
Console output for code from above:
INDEX = 1, S1 1, S2 2
INDEX = 3, S1 3, S2 4
INDEX = 5, S1 5, S2 6
INDEX = 7, S1 7, S2 8
INDEX = 9, S1 9, S2 10
INDEX = 11, S1 11, S2 12
INDEX = 13, S1 13, S2 14
INDEX = 15, S1 15, S2 16

计算这些数字的函数：

def calculate(key, dict):
temporary = []
for values in dict[key]:
temporary.append(values)
new_number = sum(temporary)
dict[key] = new_number
temporary.clear()

主

if __name__ == '__main__':
manager = multiprocessing.Manager()
dictionary = manager.dict()
fill_starting_list()
# processes = how_many_processes()
append_lists_to_dictionary((len(starting_list) // 2), dictionary)
divide_test(dictionary)
processes = []
for process_index in range(len(starting_list) // 2):
p = multiprocessing.Process(target=calculate, args=(process_index, dictionary))
p.start()
processes.append(p)
for process in processes:
process.join()

进程完成后的控制台输出：

TEST {0: 3, 1: 7, 2: 11, 3: 15, 4: 19, 5: 23, 6: 27, 7: 31, 8: 17}

Answer 1

你在寻找这样的东西吗：

import multiprocessing as mp
def fill_starting_list():
starting_list = list(range(1, 18))
print(f"Length {len(starting_list)} list {starting_list}")
return starting_list
def calculate(numbers):
return sum(numbers)
if __name__ == "__main__":

numbers = fill_starting_list()
print(f"Sum calculated by 'sum': {sum(numbers)}")
while len(numbers) > 1:
slices = (numbers[i:i + 2] for i in range(0, len(numbers), 2))
with mp.Pool() as pool:
numbers = pool.map(calculate, slices)
print(numbers)
print(f"Sum calculated by recursive multiprocessing: {numbers[0]}")

输出：

Length 17 list [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]
Sum calculated by 'sum': 153
[3, 7, 11, 15, 19, 23, 27, 31, 17]
[10, 26, 42, 58, 17]
[36, 100, 17]
[136, 17]
[153]
Sum calculated by recursive multiprocessing: 153

我在这里使用了一个进程Pool和map，而不是单独的进程，因为它要容易得多。

正如@FMc所指出的，不需要使用额外的函数caluculate，只需使用sum(也删除了fill_starting_list函数(：

import multiprocessing as mp
if __name__ == "__main__":

numbers = list(range(1, 18))
print(f"Length {len(numbers)} list {numbers}")
print(f"Sum calculated by 'sum': {sum(numbers)}")
while len(numbers) > 1:
slices = (numbers[i:i + 2] for i in range(0, len(numbers), 2))
with mp.Pool() as pool:
numbers = pool.map(sum, slices)
print(numbers)
print(f"Sum calculated by recursive multiprocessing: {numbers[0]}")