获取排序的numpy矩阵或 pandas 数据帧的最后一个非nan索引 [英] Getting the last non-nan index of a sorted numpy matrix or pandas dataframe

问题描述

给出如下的numpy数组(或pandas数据框):

Given a numpy array (or pandas dataframe) like this:

import numpy as np

a = np.array([
[1,      1,      1,    0.5, np.nan, np.nan, np.nan],
[1,      1,      1, np.nan, np.nan, np.nan, np.nan],
[1,      1,      1,    0.5,   0.25,  0.125,  0.075],
[1,      1,      1,   0.25, np.nan, np.nan, np.nan],
[1, np.nan, np.nan, np.nan, np.nan, np.nan, np.nan],
[1,      1,    0.5,    0.5, np.nan, np.nan, np.nan]
])

我正在寻找最有效地检索每一行中最后一个非nan值的方法，因此在这种情况下，我将寻找一个返回如下内容的函数:

I'm looking to most efficiently retrieve the last non-nan value in each row, so in this situation I'd be looking for a function that returns something like this:

np.array([3,
          2,
          6,
          3,
          0,
          3])

我可以尝试np.argmin(a, axis=1) - 1，但这至少具有两个不希望的属性-对于不以nan结尾的行(dealbreaker)，它会失败，并且不会延迟评估"并在到达最后一个时停止给定行中的非nan值(这与必须正确"条件无关紧要).

I can try np.argmin(a, axis=1) - 1, but this has at least two undesirable properties - it fails for rows not ending with nan (dealbreaker) and it doesn't "lazy-evaluate" and stop once it has reached the last non-nan value in a given row (this doesn't matter as much as the "it has to be right" condition).

我想有一种方法可以使用np.where进行操作，但是除了评估每一行的所有元素之外，我看不到一种明显的优雅方​​法来重新排列输出以获取每一行的最后一个索引:

I imagine there's a way to do it with np.where, but in addition to evaluating all the elements of each row, I can't see an obvious elegant way to rearrange the output to get the last index in each row:

>>> np.where(np.isnan(a))
(array([0, 0, 0, 1, 1, 1, 1, 3, 3, 3, 4, 4, 4, 4, 4, 4, 5, 5, 5]),
 array([4, 5, 6, 3, 4, 5, 6, 4, 5, 6, 1, 2, 3, 4, 5, 6, 4, 5, 6]))

推荐答案

pandas.Series具有last_valid_index方法:

pandas.Series has a last_valid_index method:

pd.DataFrame(a.T).apply(pd.Series.last_valid_index)
Out: 
0    3
1    2
2    6
3    3
4    0
5    3
dtype: int64

这篇关于获取排序的numpy矩阵或 pandas 数据帧的最后一个非nan索引的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！