我将推文中的单词限制为内容单词，现在我想将单词转换为小写并添加带有下划线的 POS

Question

我编写了下面的代码，并将推文中的单词限制为内容单词，即名词，动词和形容词，现在我想将单词转换为小写并添加带有下划线的POS。 例如：

love_VERB老fashioneds_NOUN 但我不知道怎么做，谁能帮我？

! pip install wget
import wget
url = 'https://raw.githubusercontent.com/dirkhovy/NLPclass/master/data/reviews.full.tsv.zip'
wget.download(url, 'reviews.full.tsv.zip')

from zipfile import ZipFile
with ZipFile('reviews.full.tsv.zip', 'r') as zf:
zf.extractall()

import pandas as pd
df = pd.read_csv('reviews.full.tsv', sep='t', nrows=100000) # nrows , max amount of rows 
documents = df.text.values.tolist()
print(documents[:4])

import spacy
nlp = spacy.load('en_core_web_sm') #you can use other methods
# excluded tags
included_tags = {"NOUN", "VERB", "ADJ"}
#document = [line.strip() for line in open('moby_dick.txt', encoding='utf8').readlines()]
sentences = documents[:103] #first 10 sentences
new_sentences = []
for sentence in sentences:
new_sentence = []
for token in nlp(sentence):
if token.pos_  in included_tags:
new_sentence.append(token.text)
new_sentences.append(" ".join(new_sentence))
#Creates a list of lists of tokens
tokens = [[token.text for token in nlp(new_sentence)] for new_sentence in documents[:200]]
tokens
# import itertools
# tok = itertools.chain.from_iterable(
#    [[token.text for token in nlp(new_sentence)] for new_sentence in documents[:200]])
# tok

Answer 1

我相信如果你改变

new_sentence.append(token.text)

自

new_sentence.append(token.text.lower()+'_'+token.POS)

你会得到你所追求的。