当变量是原子向量时删除负值 [英] Removing negative values when variable is an atomic vector
问题描述
我有一个庞大的调查数据集(最初是Stata(.dta)文件).我想使用下面的代码将数据集中的负值转换为NA.如果变量的NA大于99%,则代码应将其删除.
I have a large dataset of a survey (originally a Stata(.dta) file). I would like to use the code below to convert negative values in my dataset to NA. If a variable has more than 99% NA's the code should drop it.
#mixed data
WVS <- data.frame(file)
dat <- WVS[,sapply(df, function(x) {class(x)== "numeric" | class(x) == "integer"})]
# NEGATIVES -> NA
foo <- function(dat, p){
ind <- colSums(is.na(dat))/nrow(dat)
dat[dat < 0] <- NA
dat[, ind < p]
}
# process numeric part of the data separately
ii <- sapply(WVS, class) == "numeric"
WVS.num <- foo(as.matrix(WVS[, ii]), 0.99)
# then stick the two parts back together again
WVS <- data.frame(WVS[, !ii], WVS.num)
但是这似乎不起作用,因为它看起来是这样的:
This did however not work as it appears that:
> is("S004")
[1] "character" "vector" "data.frameRowLabels" "SuperClassMethod" "index"
[6] "atomicVector
Str(WVS):
$ S004 :Class 'labelled' atomic [1:50] -4 -4 -4 -4 -4 -4 -4 -4 -4 -4 ...
.. ..- attr(*, "label")= chr "Set"
.. ..- attr(*, "format.stata")= chr "%8.0g"
.. ..- attr(*, "labels")= Named num [1:7] -5 -4 -3 -2 -1 1 2
.. .. ..- attr(*, "names")= chr [1:7] "Missing; Unknown" "Not asked in survey" "Not applicable" "No answer" ...
我该如何修改我的代码以解决此问题?
How do I adapt my code to cope with this?
更新:
我更改了下面的答案,并尝试使其循环运行(因为我的数据集对于下面的解决方案而言太大.
I have altered the answer below and tried to make it function with a loop (because my dataset is too big for the solution below.
# Creating a column with the same length as the original dataset
WVSc <- data.frame(x = 1:341271, y = c(NA))
# Loop for every column
for(i in 1:ncol(WVS))
# Replace all negatives with NA if possible
{try(WVS[,i] <- NA^(WVS[,i]<0) * WVS[,i])
# Select columns to keep and create a new dataframe from these columns
col_to_keep <- sapply(WVSx[,i], function(x) sum(is.na(x)/length(x))
col_to_keep <- names(col_to_keep[col_to_keep <= 0.99])
WVSc < - cbind(WVS,col_to_keep)}
因此,以上内容实际上无效.另外,我希望通过循环删除NA大于99%的列,而不是创建一个新的df,并绑定那些较少的列.
So, the above does not really work. In addition I was hoping to, by looping, remove columns which have more than 99% NA rather than create a new df, and bind the ones that have less.
推荐答案
由于您没有提供任何示例,因此这是我的靶心解决方案.希望这会给您一些先机:
Since you haven't provided any example, here's my bulls eye solution. Hopefully this would give you some headstart:
cleanFun <- function(df){
# set negative values as NA
df[df < 0] <- NA
# faster, vectorized solution
# select numeric columns
num_cols <- names(df)[sapply(df, is.numeric)]
# get name of columns with 99% or more NA values
col_to_remove <- names(df)[colMeans(is.na(df[num_cols]))>=0.9]
# drop those columns
return (df[setdiff(colnames(df),col_to_remove)])
}
your_df <- cleanFun(your_df)
这篇关于当变量是原子向量时删除负值的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!