执行scikit-learns轮廓分数时如何修复MemoryError? [英] How can I fix a MemoryError when executing scikit-learns silhouette score?

问题描述

我运行一个聚类算法，并希望通过使用scikit-learn中的剪影得分来评估结果.但是在scikit学习中，它需要计算距离矩阵:distances = pairwise_distances(X，metric = metric，** kwds)

I run a clustering algorithm and want to evaluate the result by using silhouette score in scikit-learn. But in the scikit-learn, it needs to calculate the distance matrix: distances = pairwise_distances(X, metric=metric, **kwds)

由于我的数据约为300K，内存为2GB，结果内存不足.而且我无法评估聚类结果.

Due to the fact that my data is order of 300K, and my memory is 2GB, and the result is out of memory. And I can not evaluate the clustering result.

有人知道如何克服这个问题吗?

Does anyone know how to overcome this problem?

推荐答案

设置

Set the sample_size parameter in the call to silhouette_score to some value smaller than 300K. Using this parameter will sample datapoints from X and calculate the silhouette_score on those instead of the entire array.

这篇关于执行scikit-learns轮廓分数时如何修复MemoryError?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！