pandas 用单位插值数据 [英] Pandas interpolate data with units
问题描述
大家好,
我一直在寻找Stackoverflow两年了,它对我有很大帮助,以至于我从来不需要注册:)
I've been looking to Stackoverflow for couple of years, and it helped me a lot, so much that I never have to register before :)
但是今天,我陷入了将Python与Pandas和Quantities结合使用的问题(也可能是unum或品脱).我会尽力做出清晰的帖子,但是由于这是我的第一篇文章,因此,如果出现令人困惑的事情,我深表歉意,并将尝试纠正您会发现的任何错误:)
But today I'm stuck on a problem using Python with Pandas and Quantities (could be unum or pint as well). I try to do my best to make a clear post, but since it's my first one, I apologize if something is confusing and will try to correct any mistake you'll find :)
我想从数据源导入数据并构建一个Pandas数据框,如下所示:
I want to import data from a source and build a Pandas dataframe as follow:
import pandas as pd
import quantities as pq
depth = [0.0,1.1,2.0] * pq.m
depth2 = [0,1,1.1,1.5,2] * pq.m
s1 = pd.DataFrame(
{'depth' : [x for x in depth]},
index = depth)
这给出了:
S1=
depth
0.0 0.0 m
1.1 1.1 m
2.0 2.0 m
现在,我想将数据扩展到depth2值: (显然,没有必要在深度上插入深度,但这是在变得更加复杂之前进行的测试).
Now I want to extend the data to the depth2 values: (obviously there is not point to interpolate depth over depth, but it's a test before it gets more complicated).
s2 = s1.reindex(depth2)
这给出了:
S2=
depth
0.0 0.0 m
1.0 NaN
1.1 1.1 m
1.5 NaN
2.0 2.0 m
到目前为止没有问题.
但是当我尝试对缺失的值进行插值时:
But when I try to interpolate the missing values doing:
s2['depth'].interpolate(method='values')
我遇到以下错误:
C:\Python27\lib\site-packages\numpy\lib\function_base.pyc in interp(x, xp, fp, left, right)
1067 return compiled_interp([x], xp, fp, left, right).item()
1068 else:
-> 1069 return compiled_interp(x, xp, fp, left, right)
1070
1071
TypeError: Cannot cast array data from dtype('O') to dtype('float64') according to the rule 'safe'
我了解从numpy进行插值不适用于对象.
I understand that interpolation from numpy does not work on object.
但是,如果我现在尝试通过删除单位来插值缺失值,则可以:
But if I try now to interpolate the missing values by dropping the units, it works:
s3 = s2['depth'].astype(float).interpolate(method='values')
这给出了:
s3 =
0.0 0
1.0 1
1.1 1.1
1.5 1.5
2.0 2
Name: depth, dtype: object
如何在深度"列中找回单位?
我找不到放回单位的招数...
I can't find any trick to put back the unit...
任何帮助将不胜感激. 谢谢
Any help will be greatly appreciated. Thanks
推荐答案
好吧,我找到了一种解决方案,可能不是最好的解决方案,但是对于我的问题,它的工作原理还不错:
Ok I found a solution, might not be the best one, but for my problem it works just fine:
import pandas as pd
import quantities as pq
def extendAndInterpolate(input, newIndex):
""" Function to extend a panda dataframe and interpolate
"""
output = pd.concat([input, pd.DataFrame(index=newIndex)], axis=1)
for col in output.columns:
# (1) Try to retrieve the unit of the current column
try:
# if it succeeds, then store the unit
unit = 1 * output[col][0].units
except Exception, e:
# if it fails, which means that the column contains string
# then return 1
unit = 1
# (2) Check the type of value.
if isinstance(output[col][0], basestring):
# if it's a string return the string and fill the missing cell with this string
value = output[col].ffill()
else:
# if it's a value, to be able to interpolate, you need to:
# - (a) dump the unit with astype(float)
# - (b) interpolate the value
# - (c) add again the unit
value = [x*unit for x in output[col].astype(float).interpolate(method='values')]
#
# (3) Returned the extended pandas table with the interpolated values
output[col] = pd.Series(value, index=output.index)
# Return the output dataframe
return output
然后:
depth = [0.0,1.1,2.0] * pq.m
depth2 = [0,1,1.1,1.5,2] * pq.m
s1 = pd.DataFrame(
{'depth' : [x for x in depth]},
index = depth)
s2 = extendAndInterpolate(s1, depth2)
结果:
s1
depth
0.0 0.0 m
1.1 1.1 m
2.0 2.0 m
s2
depth
0.0 0.0 m
1.0 1.0 m
1.1 1.1 m
1.5 1.5 m
2.0 2.0 m
感谢您的帮助.
这篇关于 pandas 用单位插值数据的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!