将datetime字段添加到recarray [英] Adding datetime field to recarray

问题描述

我正在尝试将日期时间字段（datetime64）附加到现有的重新配置 - 没有太多的成功。我可以创建datetime字段，但是当我尝试将其附加到记录数组时，我得到错误：

I am trying to append a date-time field (datetime64) to an existing recarray - without much success. I can create the datetime field, but when I attempt to append it to the record array I get the error:

ValueError：解析datetime字符串时出错在位置0

ValueError: Error parsing datetime string "?" at position 0

但是，如果我将数据转换为int64，我可以添加该格式没有问题。 （代码如下）

However, if I cast the data as int64 I can add it in that format without problem. (code shown below)

任何人都知道为什么这不起作用？

Anyone know why this does not work?

（我的最终目标是写重写到一个netcdf文件，所以适当的datetime格式，考虑到这个目标也将是有帮助的）

(my ultimate goal is to write the recarray to a netcdf file, so an appropriate datetime format with that goal in mind would also be helpful)

我使用的是python 2.7.6.1，numpy 1.8.1

I am using python 2.7.6.1, numpy 1.8.1

感谢，Rob

import numpy as np
import numpy.lib.recfunctions as rf

# ----- make a recarray ---------    
dummy = np.arange(0,10)
datarray = np.core.records.fromarrays([dummy,dummy,dummy],names='a,b,c')

# ----- make some time data using datetime64 ---------
sec = np.arange(0,10)*1000
millisec = np.arange(0,10) 
mytime = sec + millisec

mytime64 = mytime.astype('timedelta64[ms]')         
basetime =  np.datetime64('1990-01-01') 
mydatetime = mytime64+basetime  

# ----- convert time data to int64 ---------
idatetime = mydatetime.astype('int64');

#------ try and append to recarray ---------
#  this works
datarray = rf.append_fields(datarray, 'iDateTime', data=idatetime)
# this doesnt
datarray = rf.append_fields(datarray, 'DateTime', data=mydatetime)

推荐答案

追溯是：

Traceback (most recent call last):
  File "stack26739733.py", line 30, in <module>
    datarray = rf.append_fields(datarray, 'DateTime', data=mydatetime, usemask=False, dtypes=mydatetime.dtype)
  File "/usr/local/lib/python2.7/site-packages/numpy/lib/recfunctions.py", line 641, in append_fields
    dtype=base.dtype.descr + data.dtype.descr)
  File "/usr/local/lib/python2.7/site-packages/numpy/ma/extras.py", line 163, in masked_all
    mask=np.ones(shape, make_mask_descr(dtype)))
  File "/usr/local/lib/python2.7/site-packages/numpy/ma/core.py", line 2644, in __new__
    _data = ndarray.view(_data, cls)
  File "/usr/local/lib/python2.7/site-packages/numpy/ma/core.py", line 2800, in __array_finalize__
    self._fill_value = _check_fill_value(None, self.dtype)
  File "/usr/local/lib/python2.7/site-packages/numpy/ma/core.py", line 402, in _check_fill_value
    dtype=ndtype,)
ValueError: Error parsing datetime string "?" at position 0

所以这个append函数构造一个掩码数组（ ma ），并检查附加的'dtype'的'fill_value'。显然 _check_fill_value 不明白 datetime dtype。看起来像是屏蔽数组和datetime之间的不兼容。

So this append function constructs a masked array (ma), and checks the 'fill_value' for the appended 'dtype'. Apparently _check_fill_value doesn't understand the datetime dtype. Looks like an incompatibility between masked array and datetime. A numpy bug report might be in order.

这是一个 numpy 简单，自己动手追加：

Here's a simple, do-it-yourself append:

dt1 = np.dtype(datarray.dtype.descr + mydatetime.dtype.descr)
newarray = np.empty(datarray.shape, dtype=dt1)
for n in datarray.dtype.names:
    newarray[n] = datarray[n]
newarray['f3'] = mydatetime

我使用union dtype构造一个空数组。然后我从字段中复制 datarray 和 mydatetime 中的数据。由于与形状相比，字段的数量通常相当小，所以该副本相当快。我很确定 rf 功能是一样的。

I construct an empty array with a union dtype. Then I copy the data from both datarray and mydatetime field by field. Since the number of fields is normally quite small compared to the shape, this copy is quite fast. I'm pretty sure the rf function does the same.

'f3'是添加字段的默认名称。您可以在创建 dt1 时更改。

'f3' is the default name of the added field. You can change that when creating dt1.

结果是：

array([(0, 0, 0, datetime.datetime(1990, 1, 1, 0, 0)),
       (1, 1, 1, datetime.datetime(1990, 1, 1, 0, 0, 1, 1000)),
       (2, 2, 2, datetime.datetime(1990, 1, 1, 0, 0, 2, 2000)),
       ...
       (9, 9, 9, datetime.datetime(1990, 1, 1, 0, 0, 9, 9000))], 
      dtype=[('a', '<i4'), ('b', '<i4'), ('c', '<i4'), ('f3', '<M8[ms]')])

将这个 newarray 相同的 _check_fill_value 错误。

np.ma.masked_array(newarray)

这篇关于将datetime字段添加到recarray的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！