如何将主导主题、贡献率和主题关键字返回到原始模型

Question

LDA Mallet主题建模有很多例子，但没有一个例子显示如何将主要主题、贡献百分比和主题关键字添加到原始数据帧中。假设这是数据集和我的代码

数据集：

Document_Id   Text
1             'Here goes one example sentence that is generic'
2             'My car drives really fast and I have no brakes'
3             'Your car is slow and needs no brakes'
4             'Your and my vehicle are both not as fast as the airplane'

代码

# Gensim
import gensim
import gensim.corpora as corpora
from gensim.utils import simple_preprocess
from gensim.models import CoherenceModel
import pandas as pd
df = pd.read_csv('data_above.csv')
data = df.Text.values.tolist() 
# Assuming I have done all the preprocessing, lemmatization and so on and ended up with data_lemmatized:
# Create Dictionary
id2word = corpora.Dictionary(data_lemmatized)
# Create Corpus
texts = data_lemmatized
# Term Document Frequency
corpus = [id2word.doc2bow(text) for text in texts]
model = gensim.models.ldamodel.LdaModel(corpus=corpus, 
id2word=id2word, 
num_topics=50,random_state=100, 
chunksize = 1000, update_every=1, 
passes=10, alpha='auto', per_word_topics=True)

我试过这样的东西，但不起作用。。。

def format_topics_sentences(ldamodel, corpus, df):
# Init output
sent_topics_df = pd.DataFrame()
# Get main topic in each document
for i, row in enumerate(ldamodel[corpus]):
row = sorted(row, key=lambda x: (x[1]), reverse=True)
# Get the Dominant topic, Perc Contribution and Keywords for each document
for j, (topic_num, prop_topic) in enumerate(row):
if j == 0:  # => dominant topic
wp = ldamodel.show_topic(topic_num)
topic_keywords = ", ".join([word for word, prop in wp])
sent_topics_df = sent_topics_df.append(pd.Series([int(topic_num), round(prop_topic,4), topic_keywords]), ignore_index=True)
else:
break
sent_topics_df.columns = ['Dominant_Topic', 'Perc_Contribution', 'Topic_Keywords']
# Add original text to the end of the output
contents = df
sent_topics_df = pd.concat([sent_topics_df, contents], axis=1)
return(sent_topics_df)

Answer 1

我在项目中也使用了这段代码。它为您提供主题关键字和每个文档中的主要主题。

要获得每个主题中文档贡献的百分比，您可以使用以下方法：

topics_docs = list()
for m in ldamallet[corpus]:
topics_docs.append(m)
topics_docs_dict = dict()
for i in range(len(df)):
topics_docs_dict[df.loc[i]["Document_Id"]] = [doc for (topic, doc) in topics_docs[i]]
topics_docs_df = pd.DataFrame(data=topics_docs_dict)
docs_topics_df = topics_docs_df.transpose()

通过上面的代码，您将在docsTopicsdf的行中有文档，在docsTtopicsdf列中有主题，并且在每个单元格中有贡献百分比。

**我的代码可以工作，但它可能不是最有效的解决方案。请编辑我的代码，如果你能使它更好或提供另一个解决方案。