从R中的data.table中删除NA [英] Remove rows with NA from data.table in R
问题描述
我想删除任何列中包含 Inf
的 data.table
的所有行。到目前为止,我一直在使用这种方法:
I'd like to remove all rows of a data.table
that contain Inf
in any of its columns. So far, I've been using this approach:
DT <- data.table(col1 = c(1,2,3), col2 = c(4,Inf,5))
DT[,drop := apply(.SD, 1, function(x) any(is.infinite(x))), by = 1:nrow(DT)]
DT <- DT[(!drop)][,drop:=NULL]
来自这个Stackoverflow问题。然而,这种方法不能很好地扩展到大量的数据。有没有更好的方法来删除 Inf
?
which comes from this Stackoverflow question. However, this approach is not well scalable to large amounts of data. Is there a better way to remove the rows with Inf
?
推荐答案
您可以使用 rowSums
来检查行的任何元素是否不是有限的。
You can use rowSums
to check if any element of a row is not finite.
DT[is.finite(rowSums(DT))]
或者你可以使用 Inf * 0
是 NA
并使用 complete.cases
OR you can use the fact that Inf * 0
is NA
and use complete.cases
DT[complete.cases(DT*0)]
有些基准测试显示 rowSums
对于较小的数据集最快, complete.cases
是较大数据集的最快解决方案。
Some benchmarking shows that the rowSums
is fastest for smaller datasets and complete.cases
is the fastest solution for larger datasets.
require(microbenchmark)
microbenchmark(
DT[is.finite(rowSums(DT))]
,
DT[complete.cases(DT*0)]
,
DT[DT[, Reduce('&', lapply(.SD, is.finite))]]
)
##
## nrow(DT) = 3000
## Unit: microseconds
## expr min lq mean median uq max neval cld
## DT[is.finite(rowSums(DT))] 786.797 839.235 864.0215 852.8465 884.756 1021.988 100 a
## DT[complete.cases(DT * 0)] 1265.658 1326.575 1363.3985 1350.0055 1386.377 1898.040 100 c
## DT[DT[, Reduce("&", lapply(.SD, is.finite))]] 1220.137 1275.030 1319.6226 1308.0555 1348.443 1624.023 100 b
##
## nrow(DT) = 300000
## Unit: milliseconds
## expr min lq mean median uq max neval cld
## DT[is.finite(rowSums(DT))] 21.617935 22.687452 26.698070 25.75765 26.07942 87.56290 100 c
## DT[complete.cases(DT * 0)] 7.209252 7.567393 9.908503 10.17569 10.37473 71.31375 100 a
## DT[DT[, Reduce("&", lapply(.SD, is.finite))]] 11.786773 12.647652 14.128624 14.78512 15.05089 15.39542 100 b
这篇关于从R中的data.table中删除NA的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!