将函数应用于文件路径列表,并将csv输出写入相应的路径



如何将函数应用于我构建的文件路径列表,并在同一路径中写入输出csv?

读取子文件夹中的文件->执行函数->写入子文件夹->转到下一个子文件夹

#opened xml by filename
with open(r'XML_opsReport 100001.xml', encoding = "utf8") as fd:
Odict_parsedFromFilePath = xmltodict.parse(fd.read()) 
#func called in func below
def activity_to_df_one_day (list_activity_this_day): 
ib_list = [pd.DataFrame(list_activity_this_day[i], columns=list_activity_this_day[i].keys()).drop("@uom") for i in range(len(list_activity_this_day))]
return pd.concat(ib_list)
#Processes parsed xml and writes csv 
def activity_to_df_all_days (Odict_parsedFromFilePath, subdir): #writes csv from parsed xml after some processing
nodes_reports = Odict_parsedFromFilePath['opsReports']['opsReport']
list_activity = []
for i in range(len(nodes_reports)):
try:
df = activity_to_df_one_day(nodes_reports[i]['activity'])
list_activity.append(df)
except KeyError:
continue
opsReport = pd.concat(list_activity)
opsReport['dTimStart'] = pd.to_datetime(opsReport['dTimStart'], infer_datetime_format =True)
opsReport.sort_values('dTimStart', axis=0, ascending=True, inplace=True, kind='quicksort', na_position='last')
opsReport.to_csv("subdiropsReport.csv") #write to the subdir


def scanfolder(): #fetches list of file-paths with desired starting name.
list_files = []
for path, dirs, files in os.walk(r'C:..xml_objects'): #directory containing several subfolders
for f in files:
if f.startswith('XML_opsReport'):
list_files.append(os.path.join(path, f))
return list_files
filepaths = scanfolder() #list of file-paths  

每个函数都能很好地工作,xml处理也很好,所以我不共享xml结构。filepaths中有100多个路径,每个路径都有不同的子目录。我希望将来也能应用上面的流,在那里我可以获得文件路径并执行所需的操作。将csv文件写入其子目录非常重要。

要获取文件所在的目录,可以使用:

import os
for root, dirs, files, in os.walk(some_dir):
for f in files:
print(root)
output_file = os.path.join(root, "output_file.csv")
print(output_file)

这就是你要找的吗?

输出:

somedir
somediroutput_file.csv

另请参阅Python 3-具有有限递归深度的travel目录树和Find current directory and file';s目录。

能够使用os.path.join解决问题。

exceptions_path_list =[]
for i in filepaths:
try:
with open(i, encoding = "utf8") as fd:
doc = xmltodict.parse(fd.read())
activity_to_df_all_days (doc, i)
except ValueError:
exceptions_path_list.append(os.path.dirname(i))
continue
def activity_to_df_all_days (Odict_parsedFromFilePath, filepath):
...
...
...    
opsReport.to_csv(os.path.join(os.path.dirname(filepath), "opsReport.csv"))

最新更新