分析Microsoft DNS调试日志

Question

我想解析Microsoft DNS调试日志响应。其想法是解析域，并打印调试日志中每个域出现的编号列表。通常，我会使用类似grep -v " R " log > tmp的东西来首先将所有响应重定向到一个文件。然后手动为像grep domain tmp这样的域grep。我想还有更好的方法。

20140416 01:38:52 588 PACKET  02030850 UDP Rcv 192.168.0.10 2659 R Q [8281   DR SERVFAIL] A     (11)quad(3)sub(7)domain(3)com(0)
20140416 01:38:52 588 PACKET  02396370 UDP Rcv 192.168.0.5 b297 R Q [8281   DR SERVFAIL] A     (3)pk(3)sub(7)domain(3)com(0)
20140415 19:46:24 544 PACKET  0261F580 UDP Snd 192.168.0.2  795a   Q [0000       NOERROR] A     (11)tertiary(7)domain(3)com(0)
20140415 19:46:24 544 PACKET  01A47E60 UDP Snd 192.168.0.1 f4e2   Q [0001   D   NOERROR] A     (11)quad(3)sub(7)domain(3)net(0)

对于上面的数据，类似于以下的输出将非常好：

domain.com 3
domain.net 1

这表明脚本或命令为domain.com找到了两个查询条目。我不担心计算中包含第三级或更高级别的主机。shell命令或Python就可以了。这里有一些伪代码，希望能把这个问题带回家。

theFile = open('log','r')
FILE = theFile.readlines()
theFile.close()
printList = []
# search for unique queries and count them
for line in FILE:
    if ('query for the " Q " field' in line):
         # store until count for this uniq value is complete
         printList.append(line)
for item in printList:
    print item    # print the summary which is a number of unique domains

Answer 1

也许是这样的？我不是正则表达式方面的专家，但这应该可以完成任务，因为我了解您正在解析的格式。

#!/usr/bin/env python
import re
ret = {}
with open('log','r') as theFile:
    for line in theFile:
        match = re.search(r'Q [.+].+(d+)([^(]+)(d+)([^(]+)',line.strip())
        if match != None:
            key = ' '.join(match.groups())
            if key not in ret.keys():
                ret[key] = 1
            else:
                ret[key] += 1
for k in ret.keys():
    print '%s %d' % (k,ret[k])

Answer 2

这个怎么样，有点暴力：

>>> from collections import Counter
>>> with open('t.txt') as f:
...     c = Counter('.'.join(re.findall(r'(w+(d+))',line.split()[-1])[-2:]) for line in f)
... 
>>> for domain, count in c.most_common():
...    print domain,count
... 
domain(3).com(0) 3
domain(3).net(0) 1

Answer 3

它不能完全满足您要求的输出，但这对您有用吗？

dns = [line.strip().split()[-1] for line in file(r"pathtofile").readlines() if "PACKET" in line]
domains = {}
for d in dns:
    if not domains.has_key(d):
        domains[d] = 1
    else:
        domains[d] += 1
for k, v in domains.iteritems():
    print "%s %d" % (k, v)