在标记化单词中计算单词频率 - 如果逻辑，则使用其他单词

Question

我正在尝试计算单词频率是数据帧中的列表。

data = {'H':[['the', 'brown', 'fox'], ['the', 'weather', 'is'],['she', 'sells', 'sea']], 'marks':['a', 'b', 'c']} 
df = pd.DataFrame(data)   

我想根据标记为 a、b、c 时分隔字数。 我知道我可以制作 x3 单独的数据帧，但我正在寻找更干净的代码输出

freq = {}
  def count_freq(word):
     for w in word:
         if w in list(freq.keys()):
            freq[w] += 1
      else:
        freq[w] = 1
df.H.apply(count_freq)

然后我试过这个，但我搞砸了

df['marks'] = z.apply(lambda row: 0 if row['marks'] in ("a")
             else if row['marks'] in ("b")
             else row['marks'] in ("c")

编辑：预期结果

            Frequency-a   Frequency-b    Frequency-c    
the         1              1
quick       1
brown       1
fox         1
she                                       1
sells                                     1
sea                                       1
weather                    1
is                         1

Answer 1

来自 sklearn MultiLabelBinarizer

from sklearn.preprocessing import MultiLabelBinarizer
mlb = MultiLabelBinarizer()
print (pd.DataFrame(mlb.fit_transform(df['H'].values),columns=mlb.classes_, index=df.marks).T)
marks    a  b  c
brown    1  0  0
fox      1  0  0
is       0  1  0
sea      0  0  1
sells    0  0  1
she      0  0  1
the      1  1  0
weather  0  1  0

Answer 2

您可以使用

crosstab unnest：

u = unnesting(df, 'H')
pd.crosstab(u.H, u.marks)

marks    a  b  c
H
brown    1  0  0
fox      1  0  0
is       0  1  0
sea      0  0  1
sells    0  0  1
she      0  0  1
the      1  1  0
weather  0  1  0

Answer 3

您可以使用

get_dummies并转置结果：

df['H'].str.join(',').str.get_dummies(sep=',').set_index(df['marks']).T
marks    a  b  c
brown    1  0  0
fox      1  0  0
is       0  1  0
sea      0  0  1
sells    0  0  1
she      0  0  1
the      1  1  0
weather  0  1  0