合并csv数据文件中数组中具有相同时间的行

Question

我有一些数据日志文件，遗憾的是，记录器选择用时间代码记录数据，时间代码以秒为单位浮动，即使每2秒记录一次。因此，在某些文件中，同一采样时间可能有几行。

日志数据启动示例：

12/1/2014 12:02:02 PM
1
Time,Notes,CO2 Pump Set Point,CO2 Pump Control Mode,CO2 Pump Alarm Set Point,CO2 Pump Flow,CO2 Pump Pressure,CO2 Pump Control Profile,CO2 Pump RPM,CO2 Pump Run State,CO2 Pump Alarm State,Cosolvent Pump Set Point,Cosolvent Pump Control Mode,Cosolvent Pump Alarm Set Point,Cosolvent Pump Flow,Cosolvent Pump Pressure,Cosolvent Pump Control Profile,Cosolvent Pump RPM,Cosolvent Pump Run State,Cosolvent Pump Alarm State,Heater Controller Zone 1 Set Point,Heater Controller Zone 2 Set Point,Heater Controller Zone 3 Set Point,Heater Controller Zone 4 Set Point,Heater Controller Zone 5 Set Point,Heater Controller Zone 6 Set Point,Heater Controller Zone 1 Alarm Set Point,Heater Controller Zone 2 Alarm Set Point,Heater Controller Zone 3 Alarm Set Point,Heater Controller Zone 4 Alarm Set Point,Heater Controller Zone 5 Alarm Set Point,Heater Controller Zone 6 Alarm Set Point,Heater Controller Zone 1 Temperature,Heater Controller Zone 2 Temperature,Heater Controller Zone 3 Temperature,Heater Controller Zone 4 Temperature,Heater Controller Zone 5 Temperature,Heater Controller Zone 6 Temperature,Heater Controller Zone 1 on/off Status,Heater Controller Zone 2 on/off Status,Heater Controller Zone 3 on/off Status,Heater Controller Zone 4 on/off Status,Heater Controller Zone 5 on/off Status,Heater Controller Zone 6 on/off Status,Heater Controller Zone 1 alarm Status,Heater Controller Zone 2 alarm Status,Heater Controller Zone 3 alarm Status,Heater Controller Zone 4 alarm Status,Heater Controller Zone 5 alarm Status,Heater Controller Zone 6 alarm Status,Pressure Regulator Set Point,Pressure Regulator Alarm Set Point,Pressure Regulator External Heater Set Point,Pressure Regulator External Heater Alarm Set Point,Pressure Regulator Valve Heater Set Point,Pressure Regulator Pressure,Pressure Regulator External Heater Temp.,Pressure Regulator Valve Heater Temp.,Pressure Regulator Run State,Pressure Regulator Needle Position,Pressure Regulator Alarm State
12:02:03:562,,50.000000,0,400,0.000000,0,2,0,0,0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
12:02:04:125,,,,,,,,,,,,,,,,,,,,0,0,0,0,,,0,0,0,0,,,0,0,0,0,,,1,1,1,1,,,0,0,0,0,,,,,,,,,,,,,
12:02:04:218,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0,400,0,100,35,0,0,26,0,3649,0
12:02:05:625,,50.000000,0,400,0.000000,0,2,0,0,0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
12:02:06:187,,,,,,,,,,,,,,,,,,,,0,0,0,0,,,0,0,0,0,,,0,0,0,0,,,1,1,1,1,,,0,0,0,0,,,,,,,,,,,,,
12:02:06:281,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0,400,0,100,35,0,0,26,0,3649,0
12:02:07:687,,50.000000,0,400,0.000000,0,2,0,0,0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
12:02:08:250,,,,,,,,,,,,,,,,,,,,0,0,0,0,,,0,0,0,0,,,0,0,0,0,,,1,1,1,1,,,0,0,0,0,,,,,,,,,,,,,
12:02:08:343,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0,400,0,100,35,0,0,27,0,3649,0

通过获取带有标题名称的数据

def timeConv(x):
    return DT.datetime.strptime(x, '%H:%M:%S:%f')
dat=genfromtxt(name,skip_header=2,delimiter=',',usecols=(2,6,32,33,34,35,50,55,59),names=True)
time=genfromtxt(name,skip_header=3,delimiter=',',usecols=(0), converters={0:timeConv},dtype=('S'))

给我的数据数组中的前三行是无效类型：

(50.0, 0.0, nan, nan, nan, nan, nan, nan, nan)
(nan, nan, 0.0, 0.0, 0.0, 0.0, nan, nan, nan)
(nan, nan, nan, nan, nan, nan, 0.0, 0.0, 3649.0)

我想遍历数据中的每一行，将dt与前一行进行比较，然后连接这些行，但我如何最有效地连接这些数据行？

for i in range(len(time)-1):
    dt=time[i+1]-time[i]
    if dt.seconds==0:
        WHAT TO PUT HERE??

Answer 1

不通过genfromtxt:获取标题名称的解决方案

dat=genfromtxt(name,skip_header=4,delimiter=',',usecols=(2,6,32,33,34,35,50,55,59))
dat=nan_to_num(dat)
c=0
idx=[]
for i in range(len(time)-1):
    dt=time[i+1]-time[i]
    if dt.seconds==0:
        if c==0:
            c=i
        dat[c]+= dat[i+1]
        idx.append(i+1)     
    else:
        c=0
dat=delete(dat,idx,axis=0)
time=delete(time,idx)

---

_{此答案发布为对问题numpy的编辑-在CC by-SA 3.0下，OP Termo从csv数据文件合并数组中具有相同时间的行。}