使用R排除观测值后在组中找到最小值的快速方法 [英] Fast way to find min in groups after excluding observations using R

问题描述

我需要对非常大的数据集（具有许多组）执行类似以下的操作，并在某处读取使用.SD的速度较慢的信息。有没有更快的方法来执行以下操作？

I need to do something similar to below on a very large data set (with many groups), and read somewhere that using .SD is slow. Is there any faster way to perform the following operation?

更准确地说，我需要创建一个新列，其中包含排除了该组观察值的子集（类似于Excel中的minif）。

To be more precise, I need to create a new column that contains the min value for each group after having excluded a subset of observations in that group (something similar to minif in Excel).

library(data.table)
dt <- data.table(valid = c(0,1,1,0,1),
                   a = c(1,1,2,3,4),
                   groups = c("A", "A", "A", "B", "B"))

dt[, valid_min := .SD[valid == 1, min(a, na.rm = TRUE)], by = groups]

输出如下：

> test
valid a k valid_min
1:     0 1 A         1
2:     1 1 A         1
3:     1 2 A         1
4:     0 3 B         4
5:     1 4 B         4

要使其更为复杂，组可能没有有效的条目或者它们可能有多个有效但缺失的条目。我当前的代码与此类似：

To make it even more complicated, groups could have no valid entries or they could have multiple valid but missing entries. My current code is similar to this:

dt <- data.table(valid = c(0,1,1,0,1,0,1,1),
                 a = c(1,1,2,3,4,3,NA,NA),
                 k = c("A", "A", "A", "B", "B", "C", "D", "D"))

dt[, valid_min := .SD[valid == 1, 
                      ifelse(all(is.na(a)), NA_real_, min(a, na.rm = TRUE))], by = k]

输出：

> dt
valid  a k valid_min
1:     0  1 A         1
2:     1  1 A         1
3:     1  2 A         1
4:     0  3 B         4
5:     1  4 B         4
6:     0  3 C        NA
7:     1 NA D        NA
8:     1 NA D        NA

推荐答案

有...

dt[dt[valid == 1 & !is.na(a), min(a), by=k], on=.(k), the_min := i.V1]

这应该很快，因为对min的内部调用已针对组进行了优化。 （请参阅？GForce 。）

This should be fast since the inner call to min is optimized for groups. (See ?GForce.)

这篇关于使用R排除观测值后在组中找到最小值的快速方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！