concurrent.futures.ProcessPoolExecutor() map 无法读取全局变量



以下是使用netmiko实现网络自动化的简单代码:所以首先我有:

  1. 函数ciscocommand用于从输入文件读取命令
  2. 函数ciscohost用于从输入文件读取主机信息
  3. 启动设备连接的函数open_connection
  4. 多处理函数run_program
  5. 函数main是主程序

所以我的代码有问题,你可以在open_connection func上看到,我们有全局变量作为commands_info,但如果我们通过多处理(run_program func(运行这个程序,则open_connectionfunc无法读取全局变量。

import time
import os
import concurrent.futures
from netmiko import ConnectHandler
from functools import partial

full_path = os.path.dirname(__file__)
host_file = os.path.join(full_path, "lab-router.txt")
command_file = os.path.join(full_path, "cisco-log.txt")
starting_time = ""

command_info = []
def cisco_command():
global command_info
command_info = []
with open(command_file, 'r') as commands:
for line in commands:
com = line.strip()
command_info.append(com)
return command_info

def cisco_host():
global starting_time
hosts_info = []
with open(host_file, 'r') as devices:
for line in devices:
deviceip = line.strip()
host = {
'device_type': 'cisco_ios',
'ip': deviceip,
'username': 'dodo',
'password': 'dodo',
'secret': 'dodo'
}
hosts_info.append(host)
starting_time = time.perf_counter()
return hosts_info
def open_connection(host):
global command_info
sendcommand = ""     
try:
connection = ConnectHandler(**host)
print('Connection Established to Host:', host['ip'])
connection.enable()
for i in command_info:
sendcommand += "n"
sendcommand += "==== {} ====".format(i)
sendcommand += "n"
sendcommand += connection.send_command(i)
sendcommand += "n"
# return sendcommand
with open("{}/{}_log.txt".format(full_path, host['ip']), 'w') as nf:
nf.write(sendcommand)  
except:
print('Connection Failed to host', host['ip'])


def run_program(hosts_info):
with concurrent.futures.ProcessPoolExecutor() as executor:
results = executor.map(open_connection, hosts_info)

for result in results:
pass
finish = time.perf_counter()
print('Time Elapsed:', finish - starting_time)

def main():
commads = cisco_command()
hosts_info = cisco_host()
run_program(hosts_info)


if __name__ == '__main__':
main()

我不知道我做错了什么

您写道:

如果我们通过多处理(run_program func(运行此程序open_connection函数无法读取全局变量。

这句话实际上是不正确的。全局变量command_info实际上是由函数open_connection读取的。这不是问题所在。

我看到您遇到的问题是,您正试图使用命令global将运行open_connection函数的每个cpu核心在command_info中所做的更改更新为主核心中的全局command_info。这行不通。假设您有4个cpu同时运行,每个cpu试图同时修改同一全局项。如果Python允许它不允许的事情,那将是一个噩梦般的场景。

实际上,Python允许您将command_info传递到每个CPU核心中(不使用全局(,您可以将该command_info视为该计算核心特有的全局变量。要将每个唯一核心的更新后的command_info传递回运行concurrent.futures.ProcessPoolExecutor()的主核心,您必须在函数open_connection(host)的末尾返回command_info(以及您的sendcommand(。然后在主核心中,您可以将其作为resultsresult中的一个术语进行访问。此后,您可以更新主核心中的全局command_info变量。

希望这个解释能帮助你理解你的问题。:(

最新更新