pandas 按持续时间拖放行 [英] Pandas drop rows by time duration

问题描述

我想按时间条件（忽略日期）删除数据帧行。我的数据包含大约1亿行。我大约有100列，每列都有不同的采样频率。

I would like to drop dataframe rows by time condition (ignoring date). My data contains around 100 million rows. I have around 100 columns and each column has different sampling frequency.

我准备了以下代码段，其中考虑了不同的采样频率：

I prepared following snippet of code that takes into account different sampling frequency:

import pandas as pd

# leave_duration=0.01 seconds
# drop_duration=0.1 seconds

i = pd.date_range('2018-01-01', periods=1000, freq='2ms')
i=i.append(pd.date_range('2018-01-01', periods=1000, freq='3ms'))
i=i.append(pd.date_range('2018-01-01', periods=1000, freq='0.5ms'))
df = pd.DataFrame({'A': range(len(i))}, index=i)
df=df.sort_index()
print(df)
# drop by duration....

在此简单示例中，数据持续约1秒钟，并具有3个不同的采样频率。目标是删除持续时间（例如0.1秒）的行，并保留持续时间（例如0.01秒）的行。我该如何使用单线？

In this simple example, there is data that lasts for around 1 second, and has 3 different sampling frequencies. The goal is to drop rows that last for (eg) 0.1 second duration and leave rows of (eg) 0.01 second duration. How can I do it with a one-liner?

推荐答案

通过 df = df.loc [' 2018-01-01 00：00：00.000000'：'2018-01-01 00：00：00.000500'] 您将拥有新的df女巫数据在 2018-01之间-01 00：00：00.000000 和 2018-01-01 00：00：00.000500
现在您可以将过滤器用于欲望日期

by df=df.loc['2018-01-01 00:00:00.000000 ':'2018-01-01 00:00:00.000500 '] you will have new df witch data are between 2018-01-01 00:00:00.000000 and 2018-01-01 00:00:00.000500 now you can apply you filter for desire dates

import pandas as pd

# leave_duration=0.01 seconds
# drop_duration=0.1 seconds

i = pd.date_range('2018-01-01', periods=1000, freq='2ms')
i=i.append(pd.date_range('2018-01-01', periods=1000, freq='3ms'))
i=i.append(pd.date_range('2018-01-01', periods=1000, freq='0.5ms'))
df = pd.DataFrame({'A': range(len(i))}, index=i)
df=df.sort_index()
print(df)

#filter data between 2018-01-01 00:00:00.000000 ':'2018-01-01 00:00:00.000500
df=df.loc['2018-01-01 00:00:00.000000 ':'2018-01-01 00:00:00.000500 ']
print(df)

输出：
是应用了数据过滤器

Output: Before data filter applied

                               A
2018-01-01 00:00:00.000000     0
2018-01-01 00:00:00.000000  2000
2018-01-01 00:00:00.000000  1000
2018-01-01 00:00:00.000500  2001
2018-01-01 00:00:00.001000  2002
...                          ...
2018-01-01 00:00:02.985000  1995
2018-01-01 00:00:02.988000  1996
2018-01-01 00:00:02.991000  1997
2018-01-01 00:00:02.994000  1998
2018-01-01 00:00:02.997000  1999

[3000 rows x 1 columns]

应用日期过滤器后：

                               A
2018-01-01 00:00:00.000000     0
2018-01-01 00:00:00.000000  2000
2018-01-01 00:00:00.000000  1000
2018-01-01 00:00:00.000500  2001

这篇关于 pandas 按持续时间拖放行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！