快速将数据从numpy复制到ctypes [英] Copy data from numpy to ctypes quickly

问题描述

我有一个在c ++和python进程之间共享的ctypes对象. python进程从该对象获取输入值，通过Tensorflow运行它们，然后剩下一个numpy数组作为输出.由于这些数组很大，我想知道是否有更好的方法将数据从tensorflow的输出复制回共享的ctypes对象，以便c ++进程可以对它们进行操作. (速度是问题，是的.)

I have a ctypes object shared between a c++ and python process. The python process takes the input values from that object, runs them through Tensorflow, and I'm left with a numpy array as output. As these arrays are pretty big I'm wondering if there is a better approach to copying the data from the output of tensorflow back to the shared ctypes object so the c++ process can act on them. (speed is the issue, yes.)

现在我要一个一个地复制每个值:

Right now I'm copying each value one by one:

output = np.array([12, 13, 11, 10]) # in reality this is fairly large (the Tensorflow result)
for i, value in enumerate(output):
    data.pressure[i] = ctypes.c_double(value)

其中data是在内存中共享的ctypes对象. (在此示例之后构造)

where data is the ctypes object shared in memory. (structured after this example)

另一方面，将数据从ctypes对象复制到numpy很容易，我想知道是否有相反的事情(从numpy到ctypes数组)，这是简单的代码:

On the other hand, copying data from the ctypes object into numpy is easy, I'm wondering if there is something that does the opposite (from numpy to the ctypes array) Here is that easy code:

# Creating a numpy array from the ctypes array
input = np.reshape(data.velocity, (1, 29791))
# Tensorflow returns a numpy array
output = sess.run(final, feed_dict={Input: input}) 
# Now how do I get the data from output into data.pressure?

作为参考，这是ctypes的外观(在python端)

For reference, this is how the ctypes looks (python side)

class TransferData(ctypes.Structure):
    _fields_ = [
        ('statusReady', ctypes.c_bool),
        # ...
        ('velocity', ctypes.c_double * (31 * 31 * 31)),
        ('pressure', ctypes.c_double * (31 * 31 * 31))
    ]

推荐答案

这显示了如何将整个数据块从numpy数组复制到ctypes数组:

This shows how to copy a whole data block from numpy array to ctypes array:

import numpy as np
import ctypes


# Preparing example

class TransferData(ctypes.Structure):
    _fields_ = [
        ('statusReady', ctypes.c_bool),
        ('velocity', ctypes.c_double * 4),
        ('pressure', ctypes.c_double * 4)
    ]


data = TransferData()
output = np.array([12., 13., 11., 10.])


# Actual code

# Both values should be equal but there could be problems with alignment settings
assert ctypes.sizeof(data.pressure) == output.nbytes

ctypes.memmove(ctypes.byref(data.pressure), output.ctypes.data, output.nbytes)


print(list(data.pressure))

这篇关于快速将数据从numpy复制到ctypes的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！