使用 csv。DictWriter 将字典的 {'key':value} 对写入.csv文件并得到" ValueError: dict 包含不在字段名称中的字段: '



我正在使用Bash来生成python脚本的输出。Bash中的字典如下所示,我发现这非常令人困惑:

error = [('Connection to DB failed', 14), ('Permission denied while closing ticket', 11), ('The ticket was modified while updating', 10), ("Ticket doesn't exist", 8), ('Timeout while retrieving information', 16), ('Tried to add information to closed ticket', 13)]

它一直让我失望,因为python IDE中的字典看起来像:

{'a':2, 'b':3, 'as':32, 'bb':13, 'da':32, 'vb':53}

我正在使用下面的代码将">错误">中的字典键和值存储到一个名为"error_message.csv">的文件中。

keys1 = ['Error','Count']

with open('error_message.csv', 'w+') as file:
writer = csv.DictWriter(file, fieldnames=keys1)
writer.writeheader()
writer.writerows([error])

我不断收到的错误是:

raise ValueError("dict contains fields not in fieldnames: "
ValueError: dict contains fields not in fieldnames: 'Timeout while retrieving information', 'Permission denied while closing ticket', 'Tried to add information to closed ticket', 'The ticket was modified while updating', 'Connection to DB failed', "Ticket doesn't exist"

在我看来,csv。DictWriter认为字典的前2个条目,即

('Connection to DB failed', 14) 
('Permission denied while closing ticket', 11)

是与字段名称 ["错误"、"计数">] 关联的字段。这意味着csv。DictWriter认为:

'Error' = ('Connection to DB failed', 14) and
'Count' = ('Permission denied while closing ticket', 11)

我已经尝试了这段代码的许多变体(包括仅使用简单的csv.writer来编写每个键,值对(,其中之一如下:

keys1 = ['Error','Count']
with open('error_message.csv', 'w+') as file:
writer = csv.DictWriter(file, fieldnames=keys1)
writer.writeheader()
for data in error:
writer.writerow(data)

使用此特定代码,我收到以下错误:

wrong_fields = rowdict.keys() - self.fieldnames
AttributeError: 'str' object has no attribute 'keys

我不确定我在哪里犯了错误。下面的代码创建error_message.csv文件。

with open('syslog.log') as file:
for row in file:
row.strip()
echeck = re.findall(r'ERROR', row)

if 'ERROR' in echeck:
echeck=re.split(r'[ ()]', row)

user=echeck[-2]
user.strip()

if user not in per_user:
per_user[user]=1
per_user[user]+=1

echeck=re.split(r'[ERROR()]',row)
errors=echeck[-3].strip()

if errors not in error:
error[errors]=1
error[errors]+=1

"syslog.log"文件包含如下所示的系统日志:

Jan 31 00:09:39 ubuntu.local ticky: INFO Created ticket [#4217] (mdouglas)
Jan 31 00:16:25 ubuntu.local ticky: INFO Closed ticket [#1754] (noel)
Jan 31 00:21:30 ubuntu.local ticky: ERROR The ticket was modified while updating (breee)
Jan 31 00:44:34 ubuntu.local ticky: ERROR Permission denied while closing ticket (ac)
Jan 31 01:00:50 ubuntu.local ticky: INFO Commented on ticket [#4709] (blossom)
Jan 31 01:29:16 ubuntu.local ticky: INFO Commented on ticket [#6518] (rr.robinson)
Jan 31 01:33:12 ubuntu.local ticky: ERROR Tried to add information to closed ticket (mcintosh)
Jan 31 01:43:10 ubuntu.local ticky: ERROR Tried to add information to closed ticket (jackowens)
Jan 31 01:49:29 ubuntu.local ticky: ERROR Tried to add information to closed ticket (mdouglas)
Jan 31 02:30:04 ubuntu.local ticky: ERROR Timeout while retrieving information (oren)
Jan 31 02:55:31 ubuntu.local ticky: ERROR Ticket doesn't exist (xlg)
Jan 31 03:05:35 ubuntu.local ticky: ERROR Timeout while retrieving information (ahmed.miller)
Jan 31 03:08:55 ubuntu.local ticky: ERROR Ticket doesn't exist (blossom)
Jan 31 03:39:27 ubuntu.local ticky: ERROR The ticket was modified while updating (bpacheco)
Jan 31 03:47:24 ubuntu.local ticky: ERROR Ticket doesn't exist (enim.non)
Jan 31 04:30:04 ubuntu.local ticky: ERROR Permission denied while closing ticket (rr.robinson)
Jan 31 04:31:49 ubuntu.local ticky: ERROR Tried to add information to closed ticket (oren)
Jan 31 04:32:49 ubuntu.local ticky: ERROR Timeout while retrieving information (mcintosh)
Jan 31 04:44:23 ubuntu.local ticky: ERROR Timeout while retrieving information (ahmed.miller)
Jan 31 04:44:46 ubuntu.local ticky: ERROR Connection to DB failed (jackowens)
Jan 31 04:49:28 ubuntu.local ticky: ERROR Permission denied while closing ticket (flavia)
Jan 31 05:12:39 ubuntu.local ticky: ERROR Tried to add information to closed ticket (oren)
Jan 31 05:18:45 ubuntu.local ticky: ERROR Tried to add information to closed ticket (sri)
Jan 31 05:23:14 ubuntu.local ticky: INFO Commented on ticket [#1097] (breee)
Jan 31 05:35:00 ubuntu.local ticky: ERROR Connection to DB failed (nonummy)
Jan 31 05:45:30 ubuntu.local ticky: INFO Created ticket [#7115] (noel)
Jan 31 05:51:30 ubuntu.local ticky: ERROR The ticket was modified while updating (flavia)
Jan 31 05:57:46 ubuntu.local ticky: INFO Commented on ticket [#2253] (nonummy)
Jan 31 06:12:02 ubuntu.local ticky: ERROR Connection to DB failed (oren)
Jan 31 06:26:38 ubuntu.local ticky: ERROR Timeout while retrieving information (xlg)
Jan 31 06:32:26 ubuntu.local ticky: INFO Created ticket [#7298] (ahmed.miller)
Jan 31 06:36:25 ubuntu.local ticky: ERROR Timeout while retrieving information (flavia)
Jan 31 06:57:00 ubuntu.local ticky: ERROR Connection to DB failed (jackowens)
Jan 31 06:59:57 ubuntu.local ticky: INFO Commented on ticket [#7255] (oren)
Jan 31 07:59:56 ubuntu.local ticky: ERROR Ticket doesn't exist (flavia)
Jan 31 08:01:40 ubuntu.local ticky: ERROR Tried to add information to closed ticket (jackowens)
Jan 31 08:03:19 ubuntu.local ticky: INFO Closed ticket [#1712] (britanni)
Jan 31 08:22:37 ubuntu.local ticky: INFO Created ticket [#2860] (mcintosh)
Jan 31 08:28:07 ubuntu.local ticky: ERROR Timeout while retrieving information (montanap)
Jan 31 08:49:15 ubuntu.local ticky: ERROR Permission denied while closing ticket (britanni)
Jan 31 08:50:50 ubuntu.local ticky: ERROR Permission denied while closing ticket (montanap)
Jan 31 09:04:27 ubuntu.local ticky: ERROR Tried to add information to closed ticket (noel)
Jan 31 09:15:41 ubuntu.local ticky: ERROR Timeout while retrieving information (oren)
Jan 31 09:18:47 ubuntu.local ticky: INFO Commented on ticket [#8385] (mdouglas)
Jan 31 09:28:18 ubuntu.local ticky: INFO Closed ticket [#2452] (jackowens)
Jan 31 09:41:16 ubuntu.local ticky: ERROR Connection to DB failed (ac)
Jan 31 10:11:35 ubuntu.local ticky: ERROR Timeout while retrieving information (blossom)
Jan 31 10:21:36 ubuntu.local ticky: ERROR Permission denied while closing ticket (montanap)
Jan 31 11:04:02 ubuntu.local ticky: ERROR Tried to add information to closed ticket (breee)
Jan 31 11:19:37 ubuntu.local ticky: ERROR Connection to DB failed (sri)
Jan 31 11:22:06 ubuntu.local ticky: ERROR Timeout while retrieving information (montanap)
Jan 31 11:31:34 ubuntu.local ticky: ERROR Permission denied while closing ticket (ahmed.miller)
Jan 31 11:40:25 ubuntu.local ticky: ERROR Connection to DB failed (mai.hendrix)
Jan 31 11:47:07 ubuntu.local ticky: INFO Commented on ticket [#4562] (ac)
Jan 31 11:58:33 ubuntu.local ticky: ERROR Tried to add information to closed ticket (ahmed.miller)
Jan 31 12:00:17 ubuntu.local ticky: INFO Created ticket [#7897] (kirknixon)
Jan 31 12:02:49 ubuntu.local ticky: ERROR Permission denied while closing ticket (mai.hendrix)
Jan 31 12:20:23 ubuntu.local ticky: ERROR Connection to DB failed (kirknixon)
Jan 31 12:20:40 ubuntu.local ticky: ERROR Ticket doesn't exist (flavia)
Jan 31 12:24:32 ubuntu.local ticky: INFO Created ticket [#5784] (sri)
Jan 31 12:50:10 ubuntu.local ticky: ERROR Permission denied while closing ticket (blossom)
Jan 31 12:58:16 ubuntu.local ticky: ERROR Tried to add information to closed ticket (nonummy)
Jan 31 13:08:10 ubuntu.local ticky: INFO Closed ticket [#8685] (rr.robinson)
Jan 31 13:48:45 ubuntu.local ticky: ERROR The ticket was modified while updating (breee)
Jan 31 14:13:00 ubuntu.local ticky: INFO Commented on ticket [#4225] (noel)
Jan 31 14:38:50 ubuntu.local ticky: ERROR The ticket was modified while updating (enim.non)
Jan 31 14:41:18 ubuntu.local ticky: ERROR Timeout while retrieving information (xlg)
Jan 31 14:45:55 ubuntu.local ticky: INFO Closed ticket [#7948] (noel)
Jan 31 14:50:41 ubuntu.local ticky: INFO Commented on ticket [#8628] (noel)
Jan 31 14:56:35 ubuntu.local ticky: ERROR Tried to add information to closed ticket (noel)
Jan 31 15:27:53 ubuntu.local ticky: ERROR Ticket doesn't exist (blossom)
Jan 31 15:28:15 ubuntu.local ticky: ERROR Permission denied while closing ticket (enim.non)
Jan 31 15:44:25 ubuntu.local ticky: INFO Closed ticket [#7333] (enim.non)
Jan 31 16:17:20 ubuntu.local ticky: INFO Commented on ticket [#1653] (noel)
Jan 31 16:19:40 ubuntu.local ticky: ERROR The ticket was modified while updating (mdouglas)
Jan 31 16:24:31 ubuntu.local ticky: INFO Created ticket [#5455] (ac)
Jan 31 16:35:46 ubuntu.local ticky: ERROR Timeout while retrieving information (oren)
Jan 31 16:53:54 ubuntu.local ticky: INFO Commented on ticket [#3813] (mcintosh)
Jan 31 16:54:18 ubuntu.local ticky: ERROR Connection to DB failed (bpacheco)
Jan 31 17:15:47 ubuntu.local ticky: ERROR The ticket was modified while updating (mcintosh)
Jan 31 17:29:11 ubuntu.local ticky: ERROR Connection to DB failed (oren)
Jan 31 17:51:52 ubuntu.local ticky: INFO Closed ticket [#8604] (mcintosh)
Jan 31 18:09:17 ubuntu.local ticky: ERROR The ticket was modified while updating (noel)
Jan 31 18:43:01 ubuntu.local ticky: ERROR Ticket doesn't exist (nonummy)
Jan 31 19:00:23 ubuntu.local ticky: ERROR Timeout while retrieving information (blossom)
Jan 31 19:20:22 ubuntu.local ticky: ERROR Timeout while retrieving information (mai.hendrix)
Jan 31 19:59:06 ubuntu.local ticky: INFO Created ticket [#6361] (enim.non)
Jan 31 20:02:41 ubuntu.local ticky: ERROR Timeout while retrieving information (xlg)
Jan 31 20:21:55 ubuntu.local ticky: INFO Commented on ticket [#7159] (ahmed.miller)
Jan 31 20:28:26 ubuntu.local ticky: ERROR Connection to DB failed (breee)
Jan 31 20:35:17 ubuntu.local ticky: INFO Created ticket [#7737] (nonummy)
Jan 31 20:48:02 ubuntu.local ticky: ERROR Connection to DB failed (mdouglas)
Jan 31 20:56:58 ubuntu.local ticky: INFO Closed ticket [#4372] (oren)
Jan 31 21:00:23 ubuntu.local ticky: INFO Commented on ticket [#2389] (sri)
Jan 31 21:02:06 ubuntu.local ticky: ERROR Connection to DB failed (breee)
Jan 31 21:20:33 ubuntu.local ticky: INFO Closed ticket [#3297] (kirknixon)
Jan 31 21:29:24 ubuntu.local ticky: ERROR The ticket was modified while updating (blossom)
Jan 31 22:58:55 ubuntu.local ticky: INFO Created ticket [#2461] (jackowens)
Jan 31 23:25:18 ubuntu.local ticky: INFO Closed ticket [#9876] (blossom)
Jan 31 23:35:40 ubuntu.local ticky: INFO Created ticket [#5896] (mcintosh)

我正在使用正则表达式在">syslog.log">文件中包含的每一行中查找单词"错误"。从所有这些行中,我继续提取错误消息以及生成错误的用户的用户名。

error={}字典存储所有不同的错误名称,而per_user={}字典存储生成ERROR的所有不同用户名。

来自 Python DictWriter 文档

创建一个对象,该对象像常规编写器一样运行,但将字典映射到输出行。fieldnames参数是一系列键,用于标识字典中传递给writerow()方法的值写入文件 f 的顺序。可选的restval参数指定字典在字段名称中缺少键时要写入的值。如果传递给 writerow(( 方法的字典包含字段名称中找不到的键,则可选的extrasaction参数指示要执行的操作。如果将其设置为默认值"raise",则会引发 ValueError。如果设置为"忽略",则会忽略字典中的额外值。任何其他可选或关键字参数都将传递到基础编写器实例。

因此,fieldnames参数是将包含在将传递给 writerow 的字典中的键。因为您传入的字典包含除ErrorCount以外的额外键,因此引发了错误。如果您有动态列,我建议您使用像 pandas 这样的库。

就您的 csv 而言,error字典是value:value而不是key:value。 您的代码只需要稍作修改即可使用此结构:使用 csv.writer 而不是 csv。字典编写器并将字典条目视为要添加到 csv 中的行。

with open('error_message.csv', 'w+') as file:
writer = csv.writer(file)
writer.writerow(keys1)
for errmsg, count in error.items():
writer.writerow([errmsg, count])

一个小的样式提示:您可以使用 defaultdict 跳过检查计数器字典中是否存在键:

import collections
error = collections.defaultdict(int)
per_user = collections.defaultdict(int)
with open('syslog.log') as file:
for row in file:
row.strip()
echeck = re.findall(r'ERROR', row)
if 'ERROR' in echeck:
echeck = re.split(r'[ ()]', row)
user = echeck[-2]
user.strip()
per_user[user] += 1
echeck = re.split(r'[ERROR()]', row)
errors = echeck[-3].strip()
error[errors] += 1

这也暴露了原始逻辑中的错误 - 对于新条目,它将设置error[errors] = 1然后立即递增,因此第一个条目将被计数两次。

最新更新