如何计算R中所有列之间的相关性并检测高度相关的变量 [英] How to compute correlations between all columns in R and detect highly correlated variables
本文介绍了如何计算R中所有列之间的相关性并检测高度相关的变量的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我有一个大数据集,包含100个变量和3000个观察值。
我想检测那些高度相关或多余的变量(列),因此删除数据框中的维数。我尝试过
,但是它仅计算一列与其他列之间的相关性;并且我总是收到错误消息
I have a big dataset with 100 variables and 3000 observations. I want to detect those variables (columns) which are highly correlated or redundant and so remove the dimensonality in the dataframe. I tried this but it calculates only the correlation between one column and the others; and I always get an error message
for(i in 1:ncol(predicteurs)){
correlations <- cor(predicteurs[,i],predicteurs[,2])
names(correlations[which.max(abs(correlations))])
}
Warning messages:
1: In cor(predicteurs[, i], predicteurs[, 2]) :
the standard deviation is zero
2: In cor(predicteurs[, i], predicteurs[, 2]) :
the standard deviation is zero
有人可以帮助我吗?
推荐答案
已针对较新的tidyverse软件包进行了更新。.
我会尝试收集相关矩阵。
Updated for newer tidyverse packages..
I would try gathering a correlation matrix.
# install.packages(c('tibble', 'dplyr', 'tidyr'))
library(tibble)
library(dplyr)
library(tidyr)
d <- data.frame(x1=rnorm(10),
x2=rnorm(10),
x3=rnorm(10))
d2 <- d %>%
as.matrix %>%
cor %>%
as.data.frame %>%
rownames_to_column(var = 'var1') %>%
gather(var2, value, -var1)
var1 var2 value
1 x1 x1 1.00000000
2 x1 x2 -0.05936703
3 x1 x3 -0.37479619
4 x2 x1 -0.05936703
5 x2 x2 1.00000000
6 x2 x3 0.43716004
7 x3 x1 -0.37479619
8 x3 x2 0.43716004
9 x3 x3 1.00000000
# .5 is an arbitrary number
filter(d2, value > .5)
# remove duplicates
d2 %>%
mutate(var_order = paste(var1, var2) %>%
strsplit(split = ' ') %>%
map_chr( ~ sort(.x) %>%
paste(collapse = ' '))) %>%
mutate(cnt = 1) %>%
group_by(var_order) %>%
mutate(cumsum = cumsum(cnt)) %>%
filter(cumsum != 2) %>%
ungroup %>%
select(-var_order, -cnt, -cumsum)
var1 var2 value
1 x1 x1 1
2 x1 x2 -0.0594
3 x1 x3 -0.375
4 x2 x2 1
5 x2 x3 0.437
6 x3 x3 1
这篇关于如何计算R中所有列之间的相关性并检测高度相关的变量的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文