给定一个词和频率对的字典，如何在scikit中进行文本挖掘

Question

我已经有了这样的词频和类别：

y = ['animals', 'restaurants', 'sports']
x = [{'cat':1, 'dog':2}, {'food':4, 'drink':2}, {'baseball':4, 'basketball':5}]

我应该如何按照教程继续构建管道，如下所示：

>>> from sklearn.pipeline import Pipeline
>>> text_clf = Pipeline([('vect', CountVectorizer()),
...                      ('tfidf', TfidfTransformer()),
...                      ('clf', MultinomialNB()),
... ])
>>> text_clf = text_clf.fit(twenty_train.data, twenty_train.target)

CountVectorizer 期待一个字符串...我想我可以从字典中创建一个字符串并重复每个单词出现的次数？

http://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_data.html

http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html

Answer 1

如果您已经有词频，请使用 DictVectorizer：

from sklearn.feature_extraction import DictVectorizer
pipeline = Pipeline([('dvect', DictVectorizer()),
                     ('tfidf', TfidfTransformer()),
                     ('clf', MultinomialNB())])
model = pipeline.fit(x, y)

然后你可以做：

>>> model.predict([{'cat':1}])[0]
'animals'