我的一个脚本遇到了一些问题...我会把问题放在子弹里。
- 问题
- /问题 1 - 将原始测试.csv与保存前修改后的测试进行比较,如果不同,则应保存,如果相同,则不应保存。
- 在我下面的代码中,数据是相同的,但由于某种原因,它认为它是不同的,我找不到为什么......
问题 - /问题 2 - 在比赛中忽略某些数据
- 我想使用
MATCH2
进行匹配,但忽略括号数据,例如在最后一个类数据中MATCH2
Mdata (D)
它需要匹配MData
问题 - 我想使用
- /问题 3 - 要匹配的切片数据
- 我想找到一种方法,所以如果我想使用
MATCH1
我可以设置MATCH1
,以便它只使用MATCH1[-1:]
最终会在这个例子中给我数字。
- 我想找到一种方法,所以如果我想使用
Testing.py
import re
import pandas
from pandas.util.testing import assert_frame_equal
# each block in the text file will be one element of this list
matchers = [[]]
i = 0
with open('testing.txt') as infile:
for line in infile:
line = line.strip()
# Blocks are seperated by blank lines
if len(line) == 0:
i += 1
matchers.append([])
# assume there are always two blank lines between items
# and just skip to the lext line
infile.next()
continue
matchers[i].append(line)
# This regular expression matches the variable number of students in each block
studentlike = re.compile('(d+) (.+) (d+/d+)')
# These are the names of the fields we expect at the end of each block
datanames = ['Data', 'misc2', 'bla3']
# We will build a table containing a list of elements for each student
table = []
for matcher in matchers:
# We use an iterator over the block lines to make indexing simpler
it = iter(matcher)
# The first two elements are match values
m1, m2 = it.next(), it.next()
# then there are a number of students
students = []
for possiblestudent in it:
m = studentlike.match(possiblestudent)
if m:
students.append(list(m.groups()))
else:
break
# After the students come the data elements, which we read into a dictionary
# We also add in the last possible student line as that didn't match the student re
dataitems = dict(item.split() for item in [possiblestudent] + list(it))
# Finally we construct the table
for student in students:
# We use the dictionary .get() method to return blanks for the missing fields
table.append([m1, m2] + student + [dataitems.get(d, '') for d in datanames])
textcols = ['MATCH2', 'MATCH1', 'TITLE01', 'MATCH3','TITLE02', 'Data', 'misc2', 'bla3']
csvdata = pandas.read_csv('testing.csv')
csvdata_old = csvdata.copy()
textdata = pandas.DataFrame(table, columns=textcols)
# Add any new columns
newCols = textdata.columns - csvdata.columns
for c in newCols:
csvdata[c] = None
mergecols = ['MATCH2', 'MATCH1', 'MATCH3']
csvdata.set_index(mergecols, inplace=True, drop=False)
csvdata_old.set_index(mergecols, inplace=True, drop=False)
textdata.set_index(mergecols, inplace=True,drop=False)
csvdata.update(textdata)
try:
assert_frame_equal(csvdata, csvdata_old)
print "True (Same)"
except:
csvdata.to_csv('testing.csv', index=False)
print "False (Different)"
testing.txt
MData
DMATCH1
3 Tommy 144512/23332
1 Jim 90000/222311
1 Elz M 90000/222311
1 Ben 90000/222311
Data $50.90
misc2 $10.40
bla3 $20.20
MData
DMATCH2
4 James Smith 2333/114441
4 Mike 90000/222311
4 Jessica Long 2333/114441
Data $50.90
bla3 $5.44
Mdata
DMATCH3
5 Joe Reane 0/0
5 Peter Jones 90000/222311
Data $10.91
misc2 $420.00
bla3 $210.00
testing.csv
MATCH1,MATCH2,TITLE,TITLE,TITLE,TITLE,TITLE,TITLE,MATCH3,DATA,TITLE,TITLE
DMATCH1,MData (N/A),data,data,data,data,data,data,Tommy,55,data,data
DMATCH1,MData (N/A),data,data,data,data,data,data,Ben,54,data,data
DMATCH1,MData (N/A),data,data,data,data,data,data,Jim,52,data,data
DMATCH1,MData (N/A),data,data,data,data,data,data,Elz M,22,data,data
DMATCH2,MData (B/B),data,data,data,data,data,data,James Smith,15,data,data
DMATCH2,MData (B/B),data,data,data,data,data,data,Jessica Long,224,data,data
DMATCH2,MData (B/B),data,data,data,data,data,data,Mike,62,data,data
DMATCH3,Mdata (D),data,data,data,data,data,data,Joe Reane,66,data,data
DMATCH3,Mdata (D),data,data,data,data,data,data,Peter Jones,256,data,data
DMATCH3,Mdata (D),data,data,data,data,data,data,Lesley Lope,5226,data,data
运行脚本后所需的testing.csv
...
MATCH1,MATCH2,TITLE,TITLE.1,TITLE.2,TITLE.3,TITLE.4,TITLE.5,MATCH3,DATA,TITLE.6,TITLE.7,Data,TITLE01,TITLE02,bla3,misc2
DMATCH1,MData (N/A),data,data,data,data,data,data,Tommy,55,data,data,$50.90,3,144512/23332,$20.20,$10.40
DMATCH1,MData (N/A),data,data,data,data,data,data,Ben,54,data,data,$50.90,1,90000/222311,$20.20,$10.40
DMATCH1,MData (N/A),data,data,data,data,data,data,Jim,52,data,data,$50.90,1,90000/222311,$20.20,$10.40
DMATCH1,MData (N/A),data,data,data,data,data,data,Elz M,22,data,data,$50.90,1,90000/222311,$20.20,$10.40
DMATCH2,MData (B/B),data,data,data,data,data,data,James Smith,15,data,data,$50.90,4,2333/114441,$5.44,
DMATCH2,MData (B/B),data,data,data,data,data,data,Jessica Long,224,data,data,$50.90,4,2333/114441,$5.44,
DMATCH2,MData (B/B),data,data,data,data,data,data,Mike,62,data,data,$50.90,4,90000/222311,$5.44,
DMATCH3,Mdata (D),data,data,data,data,data,data,Joe Reane,66,data,data,$10.91,5,0/0,$210.00,$420.00
DMATCH3,Mdata (D),data,data,data,data,data,data,Peter Jones,256,data,data,$10.91,5,90000/222311,$210.00,$420.00
DMATCH3,Mdata (D),data,data,data,data,data,data,Lesley Lope,5226,data,data,,,,,
如果有人能:),我将非常感谢您的帮助
编辑 bheklilr
testing.txt
Mdata
DMATCH3
5 Joe Reane 0/0
5 Peter Jones 90000/222311
Data $10.91
misc2 $420.00
bla3 $210.00
testing.csv
MATCH1,MATCH2,TITLE,MATCH3,DATA,TITLE
DMATCH3,Mdata (D),data,Joe Reane,66,data
DMATCH3,Mdata (D),data,Peter Jones,256,data
DMATCH3,Mdata (D),data,Lesley Lope,5226,data
运行脚本后所需的testing.csv
...
MATCH1,MATCH2,TITLE,MATCH3,DATA,TITLE.1,Data,TITLE01,TITLE02,bla3,misc2
DMATCH3,Mdata (D),data,Joe Reane,66,data,$10.91,5,0/0,$210.00,$420.00
DMATCH3,Mdata (D),data,Peter Jones,256,data,$10.91,5,90000/222311,$210.00,$420.00
DMATCH3,Mdata (D),data,Lesley Lope,5226,data,,,,,
通过您的示例,这里有一些答案:
问题1
将原始测试与保存前的修改测试进行比较.csv如果不同,则应保存,如果相同,则不应保存。
答案 1 - 它们实际上是不同的。例如,让我们输出每个 csv 在脚本末尾的样子:
# This is near the end of your script...
mergecols = ['MATCH2', 'MATCH1', 'MATCH3']
csvdata.set_index(mergecols, inplace=True, drop=False)
csvdata_old.set_index(mergecols, inplace=True, drop=False)
textdata.set_index(mergecols, inplace=True,drop=False)
csvdata.update(textdata)
# Add these 3 lines to dump your files to csv's of the same name.
csvdata.to_csv('csvdata.csv')
csvdata_old.to_csv('csvdata_old.csv')
textdata.to_csv('textdata.csv')
try:
assert_frame_equal(csvdata, csvdata_old)
print "True (Same)"
except:
print "False (Different)"
在文本编辑器或 excel 中检查生成的 csv 文件,您会发现csvdata
和csvdata_old
实际上是不同的。他们为什么不会呢。您已使用以下 python 代码部分向其添加了列:
>>> textdata.columns - csvdata.columns
Index([Data, TITLE01, TITLE02, bla3, misc2], dtype=object)
所以,你的断言是正确的。他们是不同的。
我不太确定您在问题 2 或 3 中的"匹配"是什么意思。你想匹配什么?你的意思是按类似的列对结果进行分组吗?这会是什么样子?更多信息会很棒!