列出大关联矩阵中的最高关联对? [英] List Highest Correlation Pairs from a Large Correlation Matrix in Pandas?
问题描述
如何在与熊猫相关的矩阵中找到最相关的?关于如何使用R(从Python或R 中的大型数据集中获取高度相关对的有效方法,但是我想知道如何对熊猫进行处理吗?在我的情况下,矩阵为4460x4460,因此无法从视觉上做到.
How do you find the top correlations in a correlation matrix with Pandas? There are many answers on how to do this with R (Show correlations as an ordered list, not as a large matrix or Efficient way to get highly correlated pairs from large data set in Python or R), but I am wondering how to do it with pandas? In my case the matrix is 4460x4460, so can't do it visually.
推荐答案
您可以使用DataFrame.values
获取数据的numpy数组,然后使用NumPy函数(例如argsort()
)获取相关性最高的对.
You can use DataFrame.values
to get an numpy array of the data and then use NumPy functions such as argsort()
to get the most correlated pairs.
但是,如果要在熊猫中执行此操作,则可以unstack
并对DataFrame进行排序:
But if you want to do this in pandas, you can unstack
and sort the DataFrame:
import pandas as pd
import numpy as np
shape = (50, 4460)
data = np.random.normal(size=shape)
data[:, 1000] += data[:, 2000]
df = pd.DataFrame(data)
c = df.corr().abs()
s = c.unstack()
so = s.sort_values(kind="quicksort")
print so[-4470:-4460]
以下是输出:
2192 1522 0.636198
1522 2192 0.636198
3677 2027 0.641817
2027 3677 0.641817
242 130 0.646760
130 242 0.646760
1171 2733 0.670048
2733 1171 0.670048
1000 2000 0.742340
2000 1000 0.742340
dtype: float64
这篇关于列出大关联矩阵中的最高关联对?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!