在for循环中附加数据框

Question

如果我有一个pd数据帧，其中有三列:id, start_time, end_time，并且我想将其转换为pd。df有两列:id, time

。from [001, 1, 3][002, 3, 4] to [001, 1][001, 2][001, 3][002, 3][002, 4]

目前，我正在使用for循环，并在每次迭代中附加数据帧，但它非常慢。还有其他方法可以节省时间吗?

Answer 1

如果start_time和end_time是timedelta使用:

df = pd.DataFrame([['001', 1, 3],['002', 3, 4]], 
                  columns=['id','start_time','end_time'])
print (df)
    id  start_time  end_time
0  001           1         3
1  002           3         4
#stack columns
df1 = pd.melt(df, id_vars='id', value_name='time').drop('variable', axis=1)
#convert int to timedelta 
df1['time'] = pd.to_timedelta(df1.time, unit='s')
df1.set_index('time', inplace=True)
print (df1)
           id
time         
00:00:01  001
00:00:03  002
00:00:03  001
00:00:04  002
#groupby by id and resample by one second
print (df1.groupby('id')
          .resample('1S')
          .ffill()
          .reset_index(drop=True, level=0)
          .reset_index())
      time   id
0 00:00:01  001
1 00:00:02  001
2 00:00:03  001
3 00:00:03  002
4 00:00:04  002

如果start_time和end_time是datetime使用:

df = pd.DataFrame([['001', '2016-01-01', '2016-01-03'],
                   ['002', '2016-01-03', '2016-01-04']], 
                  columns=['id','start_time','end_time'])
print (df)
    id  start_time    end_time
0  001  2016-01-01  2016-01-03
1  002  2016-01-03  2016-01-04
df1 = pd.melt(df, id_vars='id', value_name='time').drop('variable', axis=1)
#convert to datetime
df1['time'] = pd.to_datetime(df1.time)
df1.set_index('time', inplace=True)
print (df1)
             id
time           
2016-01-01  001
2016-01-03  002
2016-01-03  001
2016-01-04  002
#groupby by id and resample by one day
print (df1.groupby('id')
          .resample('1D')
          .ffill()
          .reset_index(drop=True, level=0)
          .reset_index())
        time   id
0 2016-01-01  001
1 2016-01-02  001
2 2016-01-03  001
3 2016-01-03  002
4 2016-01-04  002

Answer 2

我对你的问题的看法如下:

df.set_index('id', inplace=True)
reshaped = df.apply(lambda x: pd.Series(range(x['start time'], x['end time']+1)), axis=1).
    stack().reset_index().drop('level_1', axis=1)
reshaped.columns = ['id', 'time']
reshaped

---

<标题> 测试输入:

import pandas as pd
from io import StringIO
data = StringIO("""id,start time,end time
001, 1, 3
002, 3, 4""")
df = pd.read_csv(data, dtype={'id':'object'})
df.set_index('id', inplace=True)
print("Inn", df)
reshaped = df.apply(lambda x: pd.Series(range(x['start time'], x['end time']+1)), axis=1).
    stack().reset_index().drop('level_1', axis=1)
reshaped.columns = ['id', 'time']
print("Outn", reshaped)

输出:

In
    start time  end time
id      
001 1           3
002 3           4
Out
    id  time
0   001 1
1   001 2
2   001 3
3   002 3
4   002 4