如何在类中嵌套多处理管理器字典


我创建了一个简单的类来存储多处理值。init函数如下所示:
def __init__(self):
Manager = multiprocessing.Manager()
self.FMT = '%Y/%m/%d %H:%M:%S:%f'
self.lock = multiprocessing.Lock()
self.grabDays = Manager.dict()
self.changeAmt = 0
self.command = multiprocessing.Value(ctypes.c_wchar_p, 'start sequence')
self.get_best = multiprocessing.Value(ctypes.c_bool, False)
self.done = multiprocessing.Value(ctypes.c_bool, False)
self.status = multiprocessing.Value(ctypes.c_wchar_p, 'nothing')
self.discrepencies = Manager.dict()
self.making = Manager.dict()

但是,如果我将Manager设为self.Manager,但两者仍然等于multiprocessing.Manager(),则这一操作很好。我会在尝试将该类传递给该类无法pickle的其他函数时出错。如果我让Manager成为一个类变量,我也会得到一个错误。这对于一个字典的简单操作来说很好,但我的类中有一些函数使用嵌套字典。

例如:

Manager = multiprocessing.Manager()
num = 1
keys = list(self.grabDays.keys())
if num not in keys:
self.grabDays[num] = Manager.dict()

这不会返回错误,但是,其他进程看不到它是嵌套的,也看不到对它所做的一切。有没有一种方法可以嵌套管理器字典?

我想出了这个解决方案,老实说,它不是很优雅,但它很有效。这个嵌套的字典示例将有3个子字典,并且涉及一个时间变量。这样一来,就不会有太多难以计算的东西了。

第一步是不要将要创建的字典初始化为所创建类中的管理器字典,而是初始化为管理器列表。这看起来像这样:

def __init__(self):
self.manager = multiprocessing.Manager()
self.grabDays = self.manager.list()

很明显,你可以随便取什么名字。接下来,我创建了一个函数,将这个列表初始化为特定的字符串,这些字符串将告诉我的程序变量是什么。另外,请注意,我说的是一个3层字典,但字符串中没有第三层。这是因为第三层将是字典中的计数器,由字典创建者函数初始化。字符串中出现**的原因是,我知道在我要插入的任何变量中,一行中永远不会有2个星号。很明显,你可以选择任何你想要的分隔符,但重要的是你有一个,否则这将不起作用。

这是我的增值功能:

def addGrabDaysClass(self,time, price, slope, volume, percentage, day, hour):
self.day = day
self.hour = hour
try:    time = time.strftime('%Y/%m/%d %H:%M:%S:%f')
except: pass
self.grabDays.append(str(day)+'**'+str(hour)+'**'+str(slope)+'**'+str(price)+'**'+str(volume)+'**'+str(percentage)+'**'+str(time))

我还想有最多15天的数据收集,我已经创建了一个工作功能来做这件事,发布在下面:

import calendar
def updateGrabbedDaysDict(self):
#self.keys = self.grabDays.keys()
#del self.grabDays[self.keys[-1]]
storeToRemove = []
lister =  list(self.get(name='grabDays', model='Array'))
list2 = lister[-1].split('**')
day = float(list2[0])
time = datetime.strptime(list2[6], '%Y/%m/%d %H:%M:%S:%f')
lastYear = time.year
lastMonth = time.month -1
if lastMonth < 0:
lastMonth = 12
lastYear - 1
daysPreviousMonth = calendar.monthrange(lastYear,lastMonth)[1]
dayLookFor = day-15
if dayLookFor < 0:
dayLookFor = daysPreviousMonth + dayLookFor
for i in range(len(lister)):
list2 = lister[i].split('**')
day = float(list2[0])
if dayLookFor == day:
storeToRemove.append(list2[i])
for i in range(len(storeToRemove)):
while storeToRemove[i] in self.grabDays:
self.grabDays.remove(storeToRemove[i])

这是我创建的函数,可以调用它将信息添加到经理列表中,并确保它得到更新:

from datetime import datetime
def addGrabDays(time, price, slope, volume, percentage, arg = None):
arg.addGrabDaysClass(time, price, slope, volume, percentage, time.day, time.hour)
arg.updateGrabbedDaysDict()

最后,以下是将其返回到工作字典中的方法:

def returnData(name, arg=None):
if name == 'grabDays':  
lister =  list(arg.get(name='grabDays', model='Array'))
dictionary = {}
#self.grabDays.append(str(day)+'**'+str(hour)+'**'+str(slope)+'**'+str(price)+'**'+str(volume)+'**'+str(percentage)+'**'+str(time))
for i in range(len(lister)):
list2 = lister[i].split('**')
day = float(list2[0])
hour = float(list2[1])
slope = float(list2[2])
price = float(list2[3])
volume = float(list2[4])
percentage = float(list2[5])
time = datetime.strptime(list2[6], '%Y/%m/%d %H:%M:%S:%f')
keys = list(dictionary.keys())
if day not in keys:
dictionary[day] = {}
keys = list(dictionary[day].keys())
if hour not in keys:
dictionary[day][hour] = {}
keys = list(dictionary[day][hour].keys())
if not keys:
counter = 0
else:
counter = keys[-1]+1
dictionary[day][hour][counter] = {'slope':slope, 'price':price, 'volume':volume, 'time':time, 'percentage':percentage}
return dictionary

请注意,我从输入字符串的部分复制了注释,这样我就不必每次都上下滚动了。将代码放入一个名为storeData的python文件后,我就可以创建一个测试人员示例了。如果您仍在阅读并对基本代码感兴趣,但为了证明代码有效,这里的示例是:

代码:

from storeData import *
import multiprocessing
import time
from datetime import datetime
def try2(arg):
now = datetime.now()
for i in range(0,35):
addGrabDays(time=now, price=(i*9), slope=(i*5), volume=(i*3), percentage=(i*8), arg = arg)
time.sleep(1)
print('going to print in next process')
time.sleep(2)
print('done!')
def nextTry(arg):
time.sleep(2)
print(returnData('grabDays', arg=arg))
def foregrounder():
ult = data()
p1 = Process(target = try2, args=(ult,))
p1.start()
p2 = Process(target = nextTry, args=(ult,))
p2.start()
p1.join()
p2.join()
if __name__ == "__main__":
foregrounder()

代码结果:

going to print in next process
{14.0: {16.0: {0: {'slope': 0.0, 'price': 0.0, 'volume': 0.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 0.0}, 1: {'slope': 5.0, 'price': 9.0, 'volume': 3.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 8.0}, 2: {'slope': 10.0, 'price': 18.0, 'volume': 6.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 16.0}, 3: {'slope': 15.0, 'price': 27.0, 'volume': 9.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 24.0}, 4: {'slope': 20.0, 'price': 36.0, 'volume': 12.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 32.0}, 5: {'slope': 25.0, 'price': 45.0, 'volume': 15.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 40.0}, 6: {'slope': 30.0, 'price': 54.0, 'volume': 18.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 48.0}, 7: {'slope': 35.0, 'price': 63.0, 'volume': 21.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 56.0}, 8: {'slope': 40.0, 'price': 72.0, 'volume': 24.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 64.0}, 9: {'slope': 45.0, 'price': 81.0, 'volume': 27.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 72.0}, 10: {'slope': 50.0, 'price': 90.0, 'volume': 30.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 80.0}, 11: {'slope': 55.0, 'price': 99.0, 'volume': 33.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 88.0}, 12: {'slope': 60.0, 'price': 108.0, 'volume': 36.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 96.0}, 13: {'slope': 65.0, 'price': 117.0, 'volume': 39.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 104.0}, 14: {'slope': 70.0, 'price': 126.0, 'volume': 42.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 112.0}, 15: {'slope': 75.0, 'price': 135.0, 'volume': 45.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 120.0}, 16: {'slope': 80.0, 'price': 144.0, 'volume': 48.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 128.0}, 17: {'slope': 85.0, 'price': 153.0, 'volume': 51.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 136.0}, 18: {'slope': 90.0, 'price': 162.0, 'volume': 54.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 144.0}, 19: {'slope': 95.0, 'price': 171.0, 'volume': 57.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 152.0}, 20: {'slope': 100.0, 'price': 180.0, 'volume': 60.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 160.0}, 21: {'slope': 105.0, 'price': 189.0, 'volume': 63.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 168.0}, 22: {'slope': 110.0, 'price': 198.0, 'volume': 66.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 176.0}, 23: {'slope': 115.0, 'price': 207.0, 'volume': 69.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 184.0}, 24: {'slope': 120.0, 'price': 216.0, 'volume': 72.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 192.0}, 25: {'slope': 125.0, 'price': 225.0, 'volume': 75.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 200.0}, 26: {'slope': 130.0, 'price': 234.0, 'volume': 78.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 208.0}, 27: {'slope': 135.0, 'price': 243.0, 'volume': 81.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 216.0}, 28: {'slope': 140.0, 'price': 252.0, 'volume': 84.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 224.0}, 29: {'slope': 145.0, 'price': 261.0, 'volume': 87.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 232.0}, 30: {'slope': 150.0, 'price': 270.0, 'volume': 90.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 240.0}, 31: {'slope': 155.0, 'price': 279.0, 'volume': 93.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 248.0}, 32: {'slope': 160.0, 'price': 288.0, 'volume': 96.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 256.0}, 33: {'slope': 165.0, 'price': 297.0, 'volume': 99.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 264.0}, 34: {'slope': 170.0, 'price': 306.0, 'volume': 102.0, 'time': datetime.datetime(2020, 8, 14, 16, 8, 14, 383679), 'percentage': 272.0}}}}
done!

如果你仔细观察,你可以看到有35个实例(包括零(,它有3层。尽管如此,因为它们都是在同一时间制作的,你必须相信我,在不同的时间和天,多层仍然会像上面演示的计数器一样工作。

最新更新