将带有字符串的结构化numpy数组传递给cython函数 [英] Passing a structured numpy array with strings to a cython function

问题描述

我正在尝试在cython中创建一个函数，该函数通过定义cython结构类型来接受numpy结构化数组或记录数组.假设我有数据:

a = np.recarray(3, dtype=[('a', np.float32),  ('b', np.int32), ('c', '|S5'), ('d', '|S3')])
a[0] = (1.1, 1, 'this\0', 'to\0')
a[1] = (2.1, 2, 'that\0', 'ta\0')
a[2] = (3.1, 3, 'dogs\0', 'ot\0')

(注意:无论是否使用空终止符，都会出现以下问题)

然后我得到了cython代码:

import numpy as np
cimport numpy as np

cdef packed struct tstruct:
    np.float32_t a
    np.int32_t b
    char[5] c
    char[3] d

def test_struct(tstruct[:] x):
    cdef:
        int k
        tstruct y

    for k in xrange(3):
        y = x[k]
        print y.a, y.b, y.c, y.d

当我尝试运行test_struct(a)时，出现错误:

ValueError: Expected a dimension of size 5, got 8

如果对数组和相应的struct重新排序，以使包含字符串的字段彼此不相邻，则该函数将按预期工作.似乎Cython函数未正确检测到c和d字段之间的边界，并且好像您正在传递长度之和的char数组.

重新整理数据的简短操作(这是可能的，但并不理想)，还有另一种方法将具有固定长度字符串数据的Recarray传递给Cython吗?

更新:这似乎是潜在的Cython错误.请参阅有关Cython谷歌小组的以下讨论，以提示出现问题的地方:

https://groups.google.com/forum/#!topic/cython-users /TbLbXdi0_h4

更新2::该错误已于2014年2月23日在Github的master cython分支中得到修复，补丁计划包含在v0.20.2中:https://github.com/cython/cython/commit/58d9361e0a6d4cb3d4e87775f78e0550c2fea836

解决方案

此漏洞已于2014年2月22日在Github的master cython分支中得到修复，并计划将其包含在v0.20.2中: https://github.com/cython/cython/commit/58d9361e0a6d4cb3d4e87775f78e>

I am attempting to create a function in cython that accepts a numpy structured array or record array by defining a cython struct type. Suppose I have the data:

a = np.recarray(3, dtype=[('a', np.float32),  ('b', np.int32), ('c', '|S5'), ('d', '|S3')])
a[0] = (1.1, 1, 'this\0', 'to\0')
a[1] = (2.1, 2, 'that\0', 'ta\0')
a[2] = (3.1, 3, 'dogs\0', 'ot\0')

(Note: the problem described below occurs with or without the null terminator)

I then have the cython code:

import numpy as np
cimport numpy as np

cdef packed struct tstruct:
    np.float32_t a
    np.int32_t b
    char[5] c
    char[3] d

def test_struct(tstruct[:] x):
    cdef:
        int k
        tstruct y

    for k in xrange(3):
        y = x[k]
        print y.a, y.b, y.c, y.d

When I try to run test_struct(a), I get the error:

ValueError: Expected a dimension of size 5, got 8

If in the array and corresponding struct are reordered such that the fields containing strings are not adjacent to each other, then the function works as expected. It appears as if the Cython function is not detecting the boundary between the c and d fields correctly and thinks as if you are passing in a char array of the sum of the lengths.

Short of reshuffling the data (which is possible but not ideal), is there another way to pass a recarray with fixed length string data into Cython?

Update: This appears to be a potential Cython bug. See the following discussion on the Cython google group that hints at where the problem is arising:

https://groups.google.com/forum/#!topic/cython-users/TbLbXdi0_h4

Update 2: This bug has been fixed in the master cython branch on Github as of Feb 23, 2014 and the patch is slated for inclusion in v0.20.2: https://github.com/cython/cython/commit/58d9361e0a6d4cb3d4e87775f78e0550c2fea836

解决方案

This was a bug that has been fixed in the master cython branch on Github as of Feb 22, 2014 and the patch is slated for inclusion in v0.20.2: https://github.com/cython/cython/commit/58d9361e0a6d4cb3d4e87775f78e0550c2fea836

这篇关于将带有字符串的结构化numpy数组传递给cython函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！