下面的代码可以改得更快吗

Question

我有以下代码，当我运行它时，会有很长的停顿，然后在解析后打印一堆行，然后重复这个过程。所以我想知道为什么会有暂停，以及如何重新编码以在内存和速度方面更高效？

#!/usr/bin/env python
import os
from lxml import etree
subdirectories = []
for root, dirs, files in os.walk("/dev/shm/heatmap/OpenHPC"):
for i in dirs:
if "compute" in i:
subdirectories.append(i)
#for i in subdirectories:
#    os.system("cd /dev/shm/heatmap/OpenHPC/" + i + " && rrdtool dump cpu_temp.rrd cpu_temp.xml")
for i in subdirectories:
with open(i+"/cpu_temp.xml") as f:
doc = etree.parse(f)
it = iter(doc.xpath(
'//comment()[following-sibling::row] | //row/v/text()'
))
for db_date, db_value in zip(it, it):
with open("temperatureData.txt", 'a+') as f:
f.write(i + " " + db_date.text.strip()[0:23] + " " +  db_value + "n")
print(f.write(i + " " + db_date.text.strip()[0:23] + " " +  db_value + "n")

Answer 1

文件操作成本高昂，执行频率越低越好。我建议打开该文件一次，以便进行追加并尽量减少写入。这取决于你有多少内存可供使用，但如果你有足够的内存，我建议你构建一个条目数组，并一次性将它们批量写入文件(为了清晰起见，对一些变量进行了重命名(：

#!/usr/bin/env python
import os
from lxml import etree
subdirectories = []
for root, dirs, files in os.walk("/dev/shm/heatmap/OpenHPC"):
for subdir in dirs:
if "compute" in subdir:
subdirectories.append(subdir)
with open('temperatureData.txt', 'a+') as temperatureData:
datalines = []
for subdir in subdirectories:
with open(subdir+"/cpu_temp.xml") as f:
doc = etree.parse(f)
it = iter(doc.xpath(
'//comment()[following-sibling::row] | //row/v/text()'
))
for db_date, db_value in zip(it, it):
dataline = subdir + " " + db_data.text.strip()[0:23] + " " + db_value
datalines.append(dataline)
print(dataline)
temperatureData.write('n'.join(datalines))