在Python中打印每个层次聚类步骤中的所有聚类和样本 [英] Print all clusters and samples at each step of hierarchical clustering in python

问题描述

我使用Scipy库执行层次聚类并创建树形图。以下是简单的代码和生成的树状图：

import numpy as np
from scipy.cluster.hierarchy import dendrogram, linkage
from matplotlib import pyplot as plt

X = np.array([[5, 3],
              [10, 15],
              [15, 12],
              [24, 10],
              [30, 30],
              [85, 70],
              [71, 80],
              [60, 78],
              [70, 55],
              [80, 91]])
linkage_matrix = linkage(X, "single")
_ = dendrogram(linkage_matrix,)

我需要在集群过程的每个步骤打印属于每个集群的所有集群和样本。以下是上述数据和树状图的预期输出：

[{0}, {1}, {2}, {3}, {4}, {5}, {6}, {7}, {8}, {9}]
[{0}, {1, 2}, {3}, {4}, {5}, {6}, {7}, {8}, {9}]
[{0}, {1, 2, 3}, {4}, {5}, {6}, {7}, {8}, {9}]
[{0}, {1, 2, 3}, {4}, {5}, {6, 7}, {8}, {9}]
[{0, 1, 2, 3}, {4}, {5}, {6, 7}, {8}, {9}]
[{0, 1, 2, 3}, {4}, {5}, {6, 7, 9}, {8}]
[{0, 1, 2, 3}, {4}, {5, 6, 7, 9}, {8}]
[{0, 1, 2, 3, 4}, {5, 6, 7, 9}, {8}]
[{0, 1, 2, 3, 4}, {5, 6, 7, 9, 8}]
[{0, 1, 2, 3, 4, 5, 6, 7, 9, 8}]

请注意，如果有使用Scikit-Learn agglomerative clustering的解决方案也可以。

推荐答案

使用同一模块中的cut_tree函数，指定簇数为Cut条件。遗憾的是，在每个元素都是它自己的集群的情况下，它不会被切断，但这种情况很容易添加。

此外，从cut_tree返回的矩阵的形状是这样的，即每个列表示某个切点处的组。所以我转置了矩阵，但您也可以相应地调整for循环。

import numpy as np
from scipy.cluster.hierarchy import dendrogram, linkage, to_tree, cut_tree
from matplotlib import pyplot as plt

X = np.array([[5, 3],
              [10, 15],
              [15, 12],
              [24, 10],
              [30, 30],
              [85, 70],
              [71, 80],
              [60, 78],
              [70, 55],
              [80, 91]])
linkage_matrix = linkage(X, "single")
clusters = cut_tree(linkage_matrix, n_clusters=range(1, X.shape[0]))
print(clusters)
# insert column for the case, where every element is its own cluster
clusters = np.insert(clusters, clusters.shape[1], range(clusters.shape[0]), axis=1)
# transpose matrix
clusters = clusters.T
print(clusters)
for row in clusters[::-1]:
    # create empty dictionary
    groups = {}
    for i, g in enumerate(row):
        if g not in groups:
            # add new key to dict and assign empty set
            groups[g] = set([])
        # add to set of certain group
        groups[g].add(i)
    print(list(groups.values()))

更好的解决方案

不是两个for循环和cut_tree，而是使用集合上的操作和LINKINK_MATRATE中的信息。For循环以线性时间复杂性运行，但最耗时的将是print语句

对于大约30_000个样本，打印到文件将创建大约30 GB的大文件。

linkage_matrix = linkage(X, "single")

dct = dict([(i, {i}) for i in range(X.shape[0])])
print(list(dct.values()))
for i, row in enumerate(linkage_matrix, X.shape[0]):
    dct[i] = dct[row[0]].union(dct[row[1]])
    del dct[row[0]]
    del dct[row[1]]
    print(list(dct.values()))

这篇关于在Python中打印每个层次聚类步骤中的所有聚类和样本的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！