KL与t-SNE嵌入的分歧

Question

如何在t-SNE嵌入后得到概率分布之间的最终KL散度？我正在使用sklearn.manifold包中的TSNE函数。有什么参数我可以访问吗？

Answer 1

拟合模型有一个名为kl_diverce_的属性。（请参阅文档）。

Answer 2

您可以使用的技巧是设置TSNE函数的参数"verbose"。如果verbose=2，您将得到以下输出：

[t-SNE] Computing pairwise distances...
[t-SNE] Computing 91 nearest neighbors...
[t-SNE] Computed conditional probabilities for sample 1000 / 1262
[t-SNE] Computed conditional probabilities for sample 1262 / 1262
[t-SNE] Mean sigma: 0.000000
[t-SNE] Iteration 25: error = 0.8776680, gradient norm = 0.0074852
[t-SNE] Iteration 50: error = 0.6590481, gradient norm = 0.0073444
[t-SNE] Iteration 75: error = 0.2875383, gradient norm = 0.0039135
[t-SNE] Iteration 100: error = 0.2597728, gradient norm = 0.0032396
[t-SNE] Error after 100 iterations with early exaggeration: 0.259773
[t-SNE] Iteration 125: error = 0.2333734, gradient norm = 0.0030973
[t-SNE] Iteration 150: error = 0.2164318, gradient norm = 0.0045540
[t-SNE] Iteration 175: error = 0.2175926, gradient norm = 0.0035685
[t-SNE] Iteration 200: error = 0.2102150, gradient norm = 0.0041453
[t-SNE] Iteration 225: error = 0.2186255, gradient norm = 0.0040576
[t-SNE] Iteration 250: error = 0.2160356, gradient norm = 0.0036579
[t-SNE] Iteration 250: did not make any progress during the last 30 episodes. Finished.
[t-SNE] Error after 250 iterations: 0.216036

尽管您没有将KL的值存储为变量，但您可以比较不同的解决方案并选择KL最低的解决方案。