分组分类并计算要素中的缺失值 [英] Groupby class and count missing values in features

问题描述

我遇到了问题，即使我认为这很琐碎，也无法在Web或文档中找到任何解决方案.

I have a problem and I cannot find any solution in the web or documentation, even if I think that it is very trivial.

我想做什么?

我有一个这样的数据框

CLASS FEATURE1 FEATURE2 FEATURE3
  X      A       NaN      NaN
  X     NaN       A       NaN
  B      A        A        A

我想按标签( CLASS )分组并显示每个功能中计数的NaN值的数量，以便看起来像这样.这样做的目的是为了大致了解缺失值如何分布在不同的类上.

I want to group by the label(CLASS) and display the number of NaN-Values that are counted in every feature so that it looks like this. The purpose of this is to get a general idea how missing values are distributed over the different classes.

CLASS FEATURE1 FEATURE2 FEATURE3
  X      1        1        2
  B      0        0        0

我知道如何接收 nonnull -Values-df.groupby['CLASS'].count()

I know how to recieve the amount of nonnull-Values - df.groupby['CLASS'].count()

NaN 值是否有相似之处?

Is there something similar for the NaN-Values?

我试图从size()中减去count()，但是它返回了一个未格式化的输出，其中填充了值NaN

I tried to subtract the count() from the size() but it returned an unformatted output filled with the value NaN

推荐答案

使用isna计算遮罩，然后分组并找到总和:

Compute a mask with isna, then group and find the sum:

df.drop('CLASS', 1).isna().groupby(df.CLASS, sort=False).sum().reset_index()

  CLASS  FEATURE1  FEATURE2  FEATURE3
0     X       1.0       1.0       2.0
1     B       0.0       0.0       0.0

---

另一种选择是使用rsub沿着第0 轴从count中减去size，以进行索引对齐减法:

---

Another option is to subtract the size from the count using rsub along the 0^th axis for index aligned subtraction:

df.groupby('CLASS').count().rsub(df.groupby('CLASS').size(), axis=0)

或者，

g = df.groupby('CLASS')
g.count().rsub(g.size(), axis=0)

       FEATURE1  FEATURE2  FEATURE3
CLASS                              
B             0         0         0
X             1         1         2

---

有很多不错的答案，所以下面是一些timeits供您细读:

---

There are quite a few good answers, so here are some timeits for your perusal:

df_ = df
df = pd.concat([df_] * 10000)

%timeit df.drop('CLASS', 1).isna().groupby(df.CLASS, sort=False).sum()
%timeit df.set_index('CLASS').isna().sum(level=0)    
%%timeit
g = df.groupby('CLASS')
g.count().rsub(g.size(), axis=0)

11.8 ms ± 108 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
9.47 ms ± 379 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
6.54 ms ± 81.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

实际效果取决于您的数据和设置，因此里程可能会有所不同.

Actual performance depends on your data and setup, so your mileage may vary.

这篇关于分组分类并计算要素中的缺失值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！