如何计算BERTopic的困惑

Question

有没有办法计算BERTopic的困惑？我在BERTopic图书馆和其他地方找不到这样的东西。

Answer 1

我设法弄清楚了如何获得日志困惑，然后将其转换回

import numpy as np
model = BERTopic(top_n_words =15,
calculate_probabilities=True)
topics, probs = model.fit_transform(docs) # docs = dataset
log_perplexity = -1 * np.mean(np.log(np.sum(probs, axis=1)))
perplexity = np.exp(log_perplexity)