为什么无法对cython内存视图进行腌制？ [英] Why can't cython memory views be pickled?

问题描述

我有一个使用memoryview数组的cython模块，即...

I have a cython module that uses memoryview arrays, that is...

double[:,:] foo

我想使用多处理并行运行此模块。但是我得到了错误：

I want to run this module in parallel using multiprocessing. However I get the error:

PicklingError: Can't pickle <type 'tile_class._memoryviewslice'>: attribute lookup tile_class._memoryviewslice failed

为什么我不能选择内存视图，我该怎么做。 / p>

Why can't I pickle a memory view and what can I do about it.

推荐答案

也许通过传递实际数组而不是内存视图可以解决您的问题。
如果您想并行执行一个函数，则如果我没有记错的话，所有参数必须是可腌制的。至少python多处理就是这种情况。因此，您可以将数组传递给函数，并在函数内部创建内存视图。

Maybe passing the actual array instead of the memory view can solve your problem. If you want to execute a function in parallel, all of it parameters have to be picklable if i recall correctly. At least that is the case with python multiprocessing. So you could pass the array to the function and create the memoryview inside your function.

def some_function(matrix_as_array):
    cdef double[:,:] matrix = matrix_as_array
    ...

我不不知道这是否对您有帮助，但是我遇到了类似的问题。我将memoryview用作cdef类中的属性。我必须编写自己的 __ reduce __ 和 __ setstate __ 方法可正确释放我类的实例。通过使用 numpy.asarray 将内存视图挑选为数组并在 __ setstate __ 中恢复它对我来说很有效。我的代码的简化版本：

I don't know if this helps you, but I encountered a similar problem. I use a memoryview as an attribute in a cdef class. I had to write my own __reduce__ and __setstate__ methods to correctly unpickle instances of my class. Pickling the memory view as an array by using numpy.asarray and restoring it in __setstate__ worked for me. A reduced version of my code:

import numpy as np

cdef class Foo:
    cdef double[:,:] matrix

    def __init__(self, matrix):
        '''Assign a passed array to the typed memory view.'''
        self.matrix = matrix

    def __reduce__(self):
        '''Define how instances of Foo are pickled.'''
        d=dict()
        d['matrix'] = np.asarray(self.matrix)
        return (Foo, (d['matrix'],), d)

    def __setstate__(self, d):
        '''Define how instances of Foo are restored.'''
        self.matrix = d['matrix']

请注意， __ reduce __ 返回一个元组，该元组由可调用（ Foo ），该可调用参数的元组（即创建新 Foo实例所需的参数，在本例中为保存的矩阵）和包含恢复实例所需的所有值的字典。

Note that __reduce__ returns a tuple consisting of a callable (Foo), a tuple of parameters for that callable (i.e. what is needed to create a 'new' Foo instance, in this case the saved matrix) and the dictionary with all values needed to restore the instance.

这篇关于为什么无法对cython内存视图进行腌制？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！