t-SNE在不同的机器上产生不同的结果 [英] t-SNE generates different results on different machines

问题描述

我在100D中有大约3000个数据点，我用t-SNE投影到2D.每个数据点属于三个类之一.但是，当我在两台单独的计算机上运行脚本时，我总是得到不一致的结果.当我使用随机种子时，可能会出现一些不一致的情况，但是其中一台计算机不断取得更好的效果(我在Mac上使用Macbook Pro，而在Ubuntu上使用固定机器).

I have around 3000 datapoints in 100D that I project to 2D with t-SNE. Each datapoint belongs to one of three classes. However, when I run the script on two separate computers I keep getting inconsistent results. Some inconsistency is expected as I use a random seed, however one of the computers keeps getting better results (I use a macbook pro and a stationary machine on Ubuntu).

我使用Scikit-learn的t-SNE实现.脚本和数据是相同的，我已经手动复制了该文件夹以确保.相关代码段如下所示:

I use the t-SNE implementation from Scikit-learn. The script and data is identical, I've manually copied the folder to make sure. The relevant code snippet looks like this:

X_vectors, X_labels = self.load_data(spec_path, sound_path, subset)
tsne = TSNE(n_components=2, perplexity=25, random_state=None)
Y = tsne.fit_transform(X_vectors)
self.plot(X_labels, Y[:, 0], Y[:, 1], Y)

第一个图像是从macbook生成的一个样本，我已经运行了好几次，并且它总是在相同的x/y范围内生成相似的形状.第二个来自 Ubuntu，显然更好，我再次运行它几次以确保它继续产生更好的结果，与 mac 相比总是在更高的 x/y 范围内.不知道我在这里没有看到什么，很可能是我想念的明显东西.

The first image is one sample generated from the macbook, I've ran it several times and it always generates a similar shape within the same x/y-range. The second is from Ubuntu and is clearly better, again I've ran it several times to make sure and it continues to generate better results, always in a higher x/y-range compared to the mac. Not sure what I'm not seeing here, it may very well be something obvious that I missed.

推荐答案

TSNE是一种启发式方法.像大多数试探法一样，它的行为可能会因小变化而大不相同.这里的核心特征是:仅保证局部收敛！(不是很可靠).后者在文档中表示(遵循基本优化理论):

TSNE is a heuristic. Like most heuristics, it might behave quite differently according to small changes. The core characteristic here is: only local-convergence is guaranteed! (not very robust). The latter is indicated (follows from basic optimization-theory) in the docs:

t-SNE的成本函数不是凸的，即使用不同的初始化我们可以得到不同的结果.

t-SNE has a cost function that is not convex, i.e. with different initializations we can get different results.

虽然您已经解释说，非播种方法不是您认为的罪魁祸首(难以衡量！很难进行基准测试)，但您应该查看sklearn的版本，因为 t-sne代码是其中之一随时间变化的sklearn的活跃部分.

While you explained, that the non-seeding approach is not the culprit in your opinion (hard to measure! benchmarking is hard), you should check out your versions of sklearn, as the t-sne code is one of the more active parts of sklearn with many changes over time.

这些更改中的每一个都可能会引入与您类似的观察结果(当仅尝试一个示例时；当然，在比较t-sne实现时，更大的基准/测试集应该是更好的方法)

Each of these changes is likely to introduce observations like yours (when only trying one example; a bigger benchmark/testset should be a better approach when comparing t-sne implementations of course)

备注:其中一台计算机不断取得更好的结果:这是广泛的，因为至少有两种不同的解释:

Remark: however one of the computers keeps getting better results: this is broad as there are at least two different interpretations:

• 直观/直观地评估结果
• 看看优化后实现的 kl_divergence _

这篇关于t-SNE在不同的机器上产生不同的结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！