将pandas数据帧中的列表传递给sklearn用于TF IDF

Question

My dataframe look like this
a = pd.DataFrame({'x': {0: 'John', 1: 'Ron', 2: 'Don'}, 
'y': {0: [['Apple','Apple','Apple'],['Ball','Ball'],['Cat']], 1: [['Zebra','Zebra'],['Fox','Fox']], 2: [['Elf'],['Ball','Ball']]}})

其中"x"指文件，"y"指术语(重复出现的次数(

我想把它传给：

v = TfidfVectorizer()
z = v.fit_transform(a)

在我读取的数据中，这只会给我

z.toarray()
>array([[1.]])

这毫无意义？

Answer 1

IIUC使用列表理解来压平嵌套列表：

v = TfidfVectorizer()
z = [v.fit_transform([z for y in x for z in y]).toarray() for x in a['y']]
print (z)
[array([[1., 0., 0.],
[1., 0., 0.],
[1., 0., 0.],
[0., 1., 0.],
[0., 1., 0.],
[0., 0., 1.]]), array([[0., 1.],
[0., 1.],
[1., 0.],
[1., 0.]]), array([[0., 1.],
[1., 0.],
[1., 0.]])]