DataFrameGroupBy diff() on condition

Question

假设我有一个数据帧：

df = pd.DataFrame({'CATEGORY':['a','b','c','b','b','a','b'],
                   'VALUE':[pd.np.NaN,1,0,0,5,0,4]})

看起来像

    CATEGORY    VALUE
0      a         NaN
1      b         1
2      c         0
3      b         0
4      b         5
5      a         0
6      b         4

我把它分组：

df = df.groupby(by='CATEGORY')

现在，让我展示一下，借助一组"b"上的示例，我想要什么：

df.get_group('b')

B组：

    CATEGORY    VALUE
1      b          1
3      b          0
4      b          5
6      b          4

我需要：在每个组的范围内，计算VALUE值之间的 diff((，跳过所有 NaN s 和 0 s。所以结果应该是：

    CATEGORY    VALUE  DIFF
1      b          1      - 
3      b          0      -
4      b          5      4
6      b          4     -1

Answer 1

您可以使用diff在删除0值后减去值并NaN值：

df = pd.DataFrame({'CATEGORY':['a','b','c','b','b','a','b'],
               'VALUE':[pd.np.NaN,1,0,0,5,0,4]})
grouped = df.groupby("CATEGORY")
# define diff func
diff = lambda x: x["VALUE"].replace(0, np.NaN).dropna().diff()
df["DIFF"] = grouped.apply(diff).reset_index(0, drop=True)
print(df)
  CATEGORY  VALUE  DIFF
0        a    NaN   NaN
1        b    1.0   NaN
2        c    0.0   NaN
3        b    0.0   NaN
4        b    5.0   4.0
5        a    0.0   NaN
6        b    4.0  -1.0

Answer 2

听起来像是pd.Series.shift()操作的工作以及notnull掩码。

首先，在对数据进行分组之前，我们删除不需要的值

nonull_df = df[(df['VALUE'] != 0) & df['VALUE'].notnull()]
groups = nonull_df.groupby(by='CATEGORY')

现在我们可以在组内部移动并计算差异

nonull_df['next_value'] = groups['VALUE'].shift(1)
nonull_df['diff'] = nonull_df['VALUE'] - nonull_df['next_value']

最后，您可以选择将数据复制回原始数据帧

df.loc[nonull_df.index] = nonull_df
df
  CATEGORY  VALUE  next_value  diff
0        a    NaN         NaN   NaN
1        b    1.0         NaN   NaN
2        c    0.0         NaN   NaN
3        b    0.0         1.0  -1.0
4        b    5.0         1.0   4.0
5        a    0.0         NaN   NaN
6        b    4.0         5.0  -1.0