如何计算R中所有列之间的相关性并检测高度相关的变量 [英] How to compute correlations between all columns in R and detect highly correlated variables

问题描述

我有一个大数据集，包含100个变量和3000个观察值。
我想检测那些高度相关或多余的变量（列），因此删除数据框中的维数。我尝试过
，但是它仅计算一列与其他列之间的相关性；并且我总是收到错误消息

I have a big dataset with 100 variables and 3000 observations. I want to detect those variables (columns) which are highly correlated or redundant and so remove the dimensonality in the dataframe. I tried this but it calculates only the correlation between one column and the others; and I always get an error message

for(i in 1:ncol(predicteurs)){
correlations <- cor(predicteurs[,i],predicteurs[,2])
names(correlations[which.max(abs(correlations))])
}

  Warning messages:
 1: In cor(predicteurs[, i], predicteurs[, 2]) :
the standard deviation is zero
  2: In cor(predicteurs[, i], predicteurs[, 2]) :
 the standard deviation is zero

有人可以帮助我吗？

推荐答案

已针对较新的tidyverse软件包进行了更新。.

我会尝试收集相关矩阵。

Updated for newer tidyverse packages..

I would try gathering a correlation matrix.

# install.packages(c('tibble', 'dplyr', 'tidyr'))
library(tibble)
library(dplyr)
library(tidyr)

d <- data.frame(x1=rnorm(10),
                x2=rnorm(10),
                x3=rnorm(10))

d2 <- d %>% 
  as.matrix %>%
  cor %>%
  as.data.frame %>%
  rownames_to_column(var = 'var1') %>%
  gather(var2, value, -var1)

  var1 var2       value
1   x1   x1  1.00000000
2   x1   x2 -0.05936703
3   x1   x3 -0.37479619
4   x2   x1 -0.05936703
5   x2   x2  1.00000000
6   x2   x3  0.43716004
7   x3   x1 -0.37479619
8   x3   x2  0.43716004
9   x3   x3  1.00000000

# .5 is an arbitrary number
filter(d2, value > .5)

# remove duplicates
d2 %>%
  mutate(var_order = paste(var1, var2) %>%
           strsplit(split = ' ') %>%
           map_chr( ~ sort(.x) %>% 
                      paste(collapse = ' '))) %>%
  mutate(cnt = 1) %>%
  group_by(var_order) %>%
  mutate(cumsum = cumsum(cnt)) %>%
  filter(cumsum != 2) %>%
  ungroup %>%
  select(-var_order, -cnt, -cumsum)

  var1  var2   value
1 x1    x1     1     
2 x1    x2    -0.0594
3 x1    x3    -0.375 
4 x2    x2     1     
5 x2    x3     0.437 
6 x3    x3     1     

这篇关于如何计算R中所有列之间的相关性并检测高度相关的变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！