numpy recarray append_fields:无法追加日期时间的numpy数组 [英] numpy recarray append_fields: can't append numpy array of datetimes
问题描述
我有一个包含各种字段的recarray，我想在上面附加一个datetime对象数组.
I have a recarray containing various fields and I want to append an array of datetime objects on to it.
但是，似乎numpy.lib.recfunctions中的append_fields函数不允许我添加对象数组.
However, it seems like the append_fields function in numpy.lib.recfunctions won't let me add an array of objects.
下面是一些示例代码:
import numpy as np
import datetime
import numpy.lib.recfunctions as recfun
dtype= np.dtype([('WIND_WAVE_HGHT', '<f4'), ('WIND_WAVE_PERD', '<f4')])
obs = np.array([(0.1,10.0),(0.2,11.0),(0.3,12.0)], dtype=dtype)
dates = np.array([datetime.datetime(2001,1,1,0),
datetime.datetime(2001,1,1,0),
datetime.datetime(2001,1,1,0)])
# This doesn't work:
recfun.append_fields(obs,'obdate',dates,dtypes=np.object)
我不断收到错误TypeError: Cannot change data-type for object array.
这似乎只是np.object数组的问题，因为我可以附加其他字段.我想念什么吗?
It seems to only be an issue with np.object arrays as I can append other fields ok. Am I missing something?
推荐答案
问题
In [143]: recfun.append_fields(obs,'test',np.array([None,[],1]))
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-143-5c3de23b09f7> in <module>()
----> 1 recfun.append_fields(obs,'test',np.array([None,[],1]))
/usr/local/lib/python3.5/dist-packages/numpy/lib/recfunctions.py in append_fields(base, names, data, dtypes, fill_value, usemask, asrecarray)
615 if dtypes is None:
616 data = [np.array(a, copy=False, subok=True) for a in data]
--> 617 data = [a.view([(name, a.dtype)]) for (name, a) in zip(names, data)]
618 else:
619 if not isinstance(dtypes, (tuple, list)):
/usr/local/lib/python3.5/dist-packages/numpy/lib/recfunctions.py in <listcomp>(.0)
615 if dtypes is None:
616 data = [np.array(a, copy=False, subok=True) for a in data]
--> 617 data = [a.view([(name, a.dtype)]) for (name, a) in zip(names, data)]
618 else:
619 if not isinstance(dtypes, (tuple, list)):
/usr/local/lib/python3.5/dist-packages/numpy/core/_internal.py in _view_is_safe(oldtype, newtype)
363
364 if newtype.hasobject or oldtype.hasobject:
--> 365 raise TypeError("Cannot change data-type for object array.")
366 return
367
TypeError: Cannot change data-type for object array.
所以问题出在这个a.view([(name, a.dtype)])表达式中.它尝试从a制作单个字段结构化的数组.可以与int和str等dtypes一起使用，但对object失败.失败是核心view处理，因此不太可能改变.
So the problem is in this a.view([(name, a.dtype)]) expression. It tries to make a single field structured array from a. That works with dtypes like int and str, but fails with object. That failure is in the core view handling, so isn't likely to change.
In [148]: x=np.arange(3)
In [149]: x.view([('test', x.dtype)])
Out[149]:
array([(0,), (1,), (2,)],
dtype=[('test', '<i4')])
In [150]: x=np.array(['one','two'])
In [151]: x.view([('test', x.dtype)])
Out[151]:
array([('one',), ('two',)],
dtype=[('test', '<U3')])
In [152]: x=np.array([[1],[1,2]])
In [153]: x
Out[153]: array([[1], [1, 2]], dtype=object)
In [154]: x.view([('test', x.dtype)])
...
TypeError: Cannot change data-type for object array.
recfunctions需要单独的负载这一事实表明，它有点像死水，使用不多，也没有处于积极开发中.我没有详细检查代码，但我怀疑修复会很麻烦.
The fact that recfunctions requires a separate load indicates that it is somewhat of a backwater, that isn't used a lot, and not under active development. I haven't examined the code in detail, but I suspect a fix would be a kludge.
这是一种从头开始添加新字段的方法.它执行与append_fields相同的基本操作:
Here's a way of adding a new field from scratch. It performs the same basic actions as append_fields:
使用obs以及新的字段名称和dtype定义新的dtype:
Define a new dtype, using the obs and the new field name and dtype:
In [158]: obs.dtype.descr
Out[158]: [('WIND_WAVE_HGHT', '<f4'), ('WIND_WAVE_PERD', '<f4')]
In [159]: obs.dtype.descr+[('TEST',object)]
Out[159]: [('WIND_WAVE_HGHT', '<f4'), ('WIND_WAVE_PERD', '<f4'), ('TEST', object)]
In [160]: dt1 =np.dtype(obs.dtype.descr+[('TEST',object)])
制作一个空的目标数组，并通过按字段名称复制数据来填充它:
Make an empty target array, and fill it by copying data by field name:
In [161]: newobs = np.empty(obs.shape, dtype=dt1)
In [162]: for n in obs.dtype.names:
...: newobs[n]=obs[n]
In [167]: dates
Out[167]:
array([datetime.datetime(2001, 1, 1, 0, 0),
datetime.datetime(2001, 1, 1, 0, 0),
datetime.datetime(2001, 1, 1, 0, 0)], dtype=object)
In [168]: newobs['TEST']=dates
In [169]: newobs
Out[169]:
array([( 0.1 , 10., datetime.datetime(2001, 1, 1, 0, 0)),
( 0.2 , 11., datetime.datetime(2001, 1, 1, 0, 0)),
( 0.30000001, 12., datetime.datetime(2001, 1, 1, 0, 0))],
dtype=[('WIND_WAVE_HGHT', '<f4'), ('WIND_WAVE_PERD', '<f4'), ('TEST', 'O')])
datetime64替代
使用本机numpy日期时间，附加作品
datetime64 alternative
With the native numpy datetimes, append works
In [179]: dates64 = dates.astype('datetime64[D]')
In [180]: recfun.append_fields(obs,'test',dates64,usemask=False)
Out[180]:
array([( 0.1 , 10., '2001-01-01'),
( 0.2 , 11., '2001-01-01'), ( 0.30000001, 12., '2001-01-01')],
dtype=[('WIND_WAVE_HGHT', '<f4'), ('WIND_WAVE_PERD', '<f4'), ('test', '<M8[D]')])
append_fields有一些我的版本没有的钟声-填充值，掩码数组，recarray等.
append_fields has some bells-n-whistles that my version doesn't - fill values, masked arrays, recarray, etc.
我可以用日期创建一个结构化的数组
I could create a structured array with the dates
In [197]: sdates = np.array([(i,) for i in dates],dtype=[('test',object)])
In [198]: sdates
Out[198]:
array([(datetime.datetime(2001, 1, 1, 0, 0),),
(datetime.datetime(2001, 1, 1, 0, 0),),
(datetime.datetime(2001, 1, 1, 0, 0),)],
dtype=[('test', 'O')])
必须有一个合并现有数组字段的函数，但我找不到它.
There must be a function that merges fields of existing arrays, but I'm not finding it.
这感觉很熟悉:
https://github.com/numpy/numpy/issues/2346
这篇关于numpy recarray append_fields:无法追加日期时间的numpy数组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
