我的问题是在给定两个训练数据good_reviews.txt
和bad_reviews.txt
的情况下对文档进行分类。因此,首先,我需要加载并标记我的训练数据,其中每一行都是与审查相对应的文档。因此,我的主要任务是根据给定的测试数据对评论(行)进行分类。
我找到了一种方法,如何加载和标签名称数据如下:
from nltk.corpus import names
names = ([(name, 'male') for name in names.words('male.txt')] +
[(name, 'female') for name in names.words('female.txt')])
所以我想要的是一个类似的东西,标记lines
而不是words
。我预计代码会是这样的,当然这是不起作用的,因为.lines
是一个无效的语法:
reviews = ([(review, 'good_review') for review in reviews.lines('good_reviews.txt')] +
[(review, 'bad_review') for review in reviews.lines('bad_reviews.txt')])
我想得到这样的结果:
>>> reviews[0]
('This shampoo is very good blablabla...', 'good_review')
如果您正在读取自己的文本文件,那么与NLTK
没有太大关系,您可以简单地使用file.readlines()
:
good_reviews = """This is great!
Wow, it amazes me...
An hour of show, a lifetime of enlightment
"""
bad_reviews = """Comme si, Comme sa.
I just wasted my foo bar on this.
An hour of s**t, ****.
"""
with open('/tmp/good_reviews.txt', 'w') as fout:
fout.write(good_reviews)
with open('/tmp/bad_reviews.txt', 'w') as fout:
fout.write(bad_reviews)
reviews = []
with open('/tmp/good_reviews.txt', 'r') as fingood, open('/tmp/bad_reviews.txt', 'r') as finbad:
reviews = ([(review, 'good_review') for review in fingood.readlines()] + [(review, 'bad_review') for review in finbad.readlines()])
print reviews
[out]:
[('This is great!n', 'good_review'), ('Wow, it amazes me...n', 'good_review'), ('An hour of show, a lifetime of enlightmentn', 'good_review'), ('Comme si, Comme sa.n', 'bad_review'), ('I just wasted my foo bar on this.n', 'bad_review'), ('An hour of s**t, ****.n', 'bad_review')]
如果您要使用NLTK电影评论语料库,请参阅NLTK/Python 中使用电影评论语料库的分类