向量化Kinect现实世界中的坐标处理算法以提高速度 [英] Vectorizing the Kinect real-world coordinate processing algorithm for speed

问题描述

我最近开始使用pylibfreenect2在Linux上使用Kinect V2.

I recently started working with the Kinect V2 on Linux with pylibfreenect2.

当我第一次能够在散点图中显示深度帧数据时，我很失望地看到没有一个深度像素似乎位于正确的位置.

When I first was able to show the depth frame data in a scatter plot I was disappointed to see that none of the depth pixels seemed to be in the correct location.

房间的侧视图(注意天花板是弯曲的).

我做了一些研究，发现其中涉及一些简单的触发操作来进行转换.

I did some research and realized there's some simple trig involved to do the conversions.

为了进行测试，我从pylibfreenect2中的预编写函数开始，该函数接受列，行和深度像素强度，然后返回该像素的实际位置:

To test I started with a pre-written function in pylibfreenect2 which accepts a column, row and a depth pixel intensity then returns that pixel's actual position:

X, Y, Z = registration.getPointXYZ(undistorted, row, col)

这在纠正位置方面表现出奇的出色:

使用 getPointXYZ()或 getPointXYZRGB()的唯一缺点是它们一次只能处理一个像素.在Python中这可能需要一段时间，因为它需要使用嵌套的for循环，如下所示:

The only drawback to using getPointXYZ() or getPointXYZRGB() is that they work on only one pixel at a time. This can take a while in Python as it requires the use of nested for-loops like so:

n_rows = d.shape[0]
n_columns = d.shape[1]
out = np.zeros((n_rows * n_columns, 3), dtype=np.float64)
for row in range(n_rows):
    for col in range(n_columns):
        X, Y, Z = registration.getPointXYZ(undistorted, row, col)
        out[row * n_columns + col] = np.array([Z, X, -Y])

我试图更好地了解getPointXYZ()是如何计算坐标的. 据我所知，它看起来类似于此OpenKinect-for-Processing函数:

I tried to better understand how getPointXYZ() was calculating a coordinate. To the best of my knowlege it looks similar to this OpenKinect-for-Processing function: depthToPointCloudPos(). Though I suspect libfreenect2's version has more going on under the hood.

以该gitHub源代码为例，然后我尝试用Python重新编写它以进行自己的实验，并提出以下内容:

Using that gitHub sourcecode as an example I then tried to re-write it in Python for my own experimentation and came up whth the following:

#camera information based on the Kinect v2 hardware
CameraParams = {
  "cx":254.878,
  "cy":205.395,
  "fx":365.456,
  "fy":365.456,
  "k1":0.0905474,
  "k2":-0.26819,
  "k3":0.0950862,
  "p1":0.0,
  "p2":0.0,
}

def depthToPointCloudPos(x_d, y_d, z, scale = 1000):
    #calculate the xyz camera position based on the depth data    
    x = (x_d - CameraParams['cx']) * z / CameraParams['fx']
    y = (y_d - CameraParams['cy']) * z / CameraParams['fy']

    return x/scale, y/scale, z/scale

这是传统的getPointXYZ和我的自定义函数之间的比较:

它们看起来非常相似.但是有明显的差异.左边的比较显示出平直的天花板上的边缘更加笔直，也有一些正弦曲线形状.我怀疑还涉及其他数学.

They look very similar. However with apparent differences. The left comparison shows edges that are more straight also some sinusoid shape on the flat ceiling. I suspect that additional math is involved.

如果有人对我的函数和libfreenect2的getPointXYZ之间的区别有任何想法，我将非常感兴趣.

I would be very interested to hear if anyone has ideas as to what might differ between my function and libfreenect2's getPointXYZ.

但是，我在这里发布的主要原因是要询问有关尝试对上述函数进行向量化以在整个数组上工作而不是遍历每个元素的问题.

However the main reason I've posted here is to ask about attempting to vectorize the above function to work on an entire array instead of looping through each element.

应用我从以上中学到的知识，我能够编写一个函数，该函数似乎是depthToPointCloudPos的矢量替代:

Applying what I learned from the above I was able to write a function which appears to be a vectorized alternative to depthToPointCloudPos:

感谢本杰明帮助使此功能更加有效！

def depthMatrixToPointCloudPos(z, scale=1000):
    #bacically this is a vectorized version of depthToPointCloudPos()
    C, R = np.indices(z.shape)

    R = np.subtract(R, CameraParams['cx'])
    R = np.multiply(R, z)
    R = np.divide(R, CameraParams['fx'] * scale)

    C = np.subtract(C, CameraParams['cy'])
    C = np.multiply(C, z)
    C = np.divide(C, CameraParams['fy'] * scale)

    return np.column_stack((z.ravel() / scale, R.ravel(), -C.ravel()))

这有效并产生与先前函数depthToPointCloudPos()相同的点云结果.唯一的区别是我的处理速率从〜1 Fps升至5-10 Fps(WhooHoo！).我相信这消除了Python执行所有计算所造成的瓶颈.这样我的散点图现在可以再次平稳运行，并且正在计算半真实世界的坐标.

This works and produces the same pointcloud results as the previous function depthToPointCloudPos(). The only difference is that my processing rate went from ~1 Fps to 5-10 Fps (WhooHoo!). I believe this eliminates a bottle-neck caused by Python doing all the calculations. So my scatter plot now runs smoothly again with the semi-real-world coordinates being calculated.

现在，我具有从深度框中检索3d坐标的高效功能，我真的很想将这种方法应用于将彩色相机数据映射到我的深度像素上.但是，我不确定要进行哪些数学运算或变量运算，也没有太多提及如何在Google上进行计算.

Now that I have an efficient function for retrieving the 3d coordinates from the depth frame, I would really like to apply this approach to also mapping the color camera data to my depth pixels. However I am not sure what math or variables are involved to do that, and there was not much mention about how to calculate it on Google.

或者，我能够使用libfreenect2使用getPointXYZRGB将颜色映射到深度像素:

Alternatively I was able to use libfreenect2 to map the colors to my depth pixels using getPointXYZRGB:

#Format undistorted and regisered data to real-world coordinates with mapped colors (dont forget color=out_col in setData)
n_rows = d.shape[0]
n_columns = d.shape[1]
out = np.zeros((n_rows * n_columns, 3), dtype=np.float64)
colors = np.zeros((d.shape[0] * d.shape[1], 3), dtype=np.float64)
for row in range(n_rows):
    for col in range(n_columns):
        X, Y, Z, B, G, R = registration.getPointXYZRGB(undistorted, registered, row, col)
        out[row * n_columns + col] = np.array([X, Y, Z])
        colors[row * n_columns + col] = np.divide([R, G, B], 255)
sp2.setData(pos=np.array(out, dtype=np.float64), color=colors, size=2)

产生点云和彩色顶点(非常慢< 1Fps):

总而言之，我的两个问题基本上是:

In summary my two questions basically are:

• 需要执行哪些附加步骤，才能使从我的 depthToPointCloudPos()函数(以及矢量化实现)返回的真实3d坐标数据更类似于getPointXYZ返回的数据()来自libfreenect2?

• What additional steps would be required so that the real-world 3d coordinates data returned from my depthToPointCloudPos() function (and the vectorized implementation) are more resemblant of the data returned by getPointXYZ() from libfreenect2?

而且，在我自己的应用程序中创建一种(可能是矢量化的)方式来生成深度到颜色配准图的方法会涉及到什么?请查看此更新，因为此问题已解决.

And, what would be involved in creating a (possibly vectorized) way to generate the depth to color registration map in my own application? Please see the update as this has been solved.

[更新]

我设法使用注册的帧将颜色数据映射到每个像素. 这非常简单，只需要在调用setData()之前添加以下行即可:

I managed to map the color data to each pixel using the registered frame. It was very simple and only required adding these lines prior to calling setData():

colors = registered.asarray(np.uint8)
colors = np.divide(colors, 255)
colors = colors.reshape(colors.shape[0] * colors.shape[1], 4 )
colors = colors[:, :3:] #BGRA to BGR (slices out the alpha channel)  
colors = colors[...,::-1] #BGR to RGB

这使Python可以快速处理颜色数据并提供平滑的结果.我已经将它们更新/添加到下面的功能示例中.

This allows Python to quickly process the color data and gives smooth results. I have updated/added them to the functional example below.

使用实时在Python中进行颜色配准的现实坐标处理！

(GIF图像分辨率已大大降低)

[更新]

花了更多时间在应用程序上之后，我添加了一些附加参数并调整了它们的值，以期改善散点图的视觉质量，并可能使此示例/问题的内容更加直观.

After spending a little more time with the application I have added some additional parameters and tuned their values with hopes to improve the visual quality of the scatter plot and possibly make things more intuitive for this example/question.

最重要的是，我将顶点设置为不透明:

Most importantly I have set the vertexes to be opaque:

sp2 = gl.GLScatterPlotItem(pos=pos)
sp2.setGLOptions('opaque') # Ensures not to allow vertexes located behinde other vertexes to be seen.

然后我注意到，每当非常接近曲面进行缩放时，相邻顶点之间的距离就会扩大，直到所有可见的空间几乎都是空白为止.部分原因是顶点的点大小未更改.

I then noticed whenever zooming very close to surfaces, the distance between adjacent verts would appear to expand until all of what was visible was mostly empty space. This was partially a result of the point size of the vertexes not changing.

为帮助创建充满彩色顶点的缩放友好"视口，我添加了以下行，这些行根据当前缩放级别(每次更新)计算顶点的大小:

To help aid in creating a "zoom-friendly" viewport full of colored vertexes I added these lines which calculates the vertex point size based on the current zoom level (for each update):

# Calculate a dynamic vertex size based on window dimensions and camera's position - To become the "size" input for the scatterplot's setData() function.
v_rate = 8.0 # Rate that vertex sizes will increase as zoom level increases (adjust this to any desired value).
v_scale = np.float32(v_rate) / gl_widget.opts['distance'] # Vertex size increases as the camera is "zoomed" towards center of view.
v_offset = (gl_widget.geometry().width() / 1000)**2 # Vertex size is offset based on actual width of the viewport.
v_size = v_scale + v_offset

瞧，

(同样，GIF图像分辨率也大大降低了)

也许不如将Pointcloud蒙皮好，但是在尝试了解您实际正在看的内容时，它确实有助于使事情变得更容易.

Maybe not quite as good as skinning a pointcloud, but it does seem to help make things easier when trying to understand what you're actually looking at.

所有提到的修改都已包含在功能示例中.

[更新]

如前两个动画所示，很明显，与坐标轴相比，真实坐标系的点云具有偏斜的方向.这是因为我没有用真实的词来补偿Kinect的实际方向！

As seen in the previous two animations it is clear that the pointcloud of real-world coordinates has a skewed orientation compared to the grid axes. This is because I was not compensating for the Kinect's actual orientation in the real word!

因此，我实现了一个附加的矢量化三角函数，该函数为每个顶点计算一个新的(旋转的和偏移的)坐标.这使它们相对于Kinect在实际空间中的实际位置正确定向.并且在使用倾斜的三脚架时很有必要(也可用于连接INU或陀螺仪/加速度计的输出以进行实时反馈.

Thus I have implemented an additional vectorized trig function which calculates a new (rotated and offset) coordinate for each vertex. This orients them correctly relative to the Kinect's actual position in real space. And is necessary when using tripods that tilt (could also be used to connect the output of an INU or gyro/accelerometer for real-time feedback).

def applyCameraMatrixOrientation(pt):
    # Kinect Sensor Orientation Compensation
    # bacically this is a vectorized version of applyCameraOrientation()
    # uses same trig to rotate a vertex around a gimbal.
    def rotatePoints(ax1, ax2, deg):
        # math to rotate vertexes around a center point on a plane.
        hyp = np.sqrt(pt[:, ax1] ** 2 + pt[:, ax2] ** 2) # Get the length of the hypotenuse of the real-world coordinate from center of rotation, this is the radius!
        d_tan = np.arctan2(pt[:, ax2], pt[:, ax1]) # Calculate the vertexes current angle (returns radians that go from -180 to 180)

        cur_angle = np.degrees(d_tan) % 360 # Convert radians to degrees and use modulo to adjust range from 0 to 360.
        new_angle = np.radians((cur_angle + deg) % 360) # The new angle (in radians) of the vertexes after being rotated by the value of deg.

        pt[:, ax1] = hyp * np.cos(new_angle) # Calculate the rotated coordinate for this axis.
        pt[:, ax2] = hyp * np.sin(new_angle) # Calculate the rotated coordinate for this axis.

    #rotatePoints(1, 2, CameraPosition['roll']) #rotate on the Y&Z plane # Disabled because most tripods don't roll. If an Inertial Nav Unit is available this could be used)
    rotatePoints(0, 2, CameraPosition['elevation']) #rotate on the X&Z plane
    rotatePoints(0, 1, CameraPosition['azimuth']) #rotate on the X&Y plane

    # Apply offsets for height and linear position of the sensor (from viewport's center)
    pt[:] += np.float_([CameraPosition['x'], CameraPosition['y'], CameraPosition['z']])



    return pt

仅需注意:只为高程"和方位角"调用rotatePoints().这是因为大多数三脚架都不支持滚动，并且为了节省CPU周期，默认情况下已将其禁用.如果您打算做一些花哨的事情，那么绝对可以取消评论！

请注意，此图像中的网格地板水平，但左点云未与之对齐:

用于设置Kinect方向的参数:

The parameters to set the Kinect's orientation:

CameraPosition = {
    "x": 0, # actual position in meters of kinect sensor relative to the viewport's center.
    "y": 0, # actual position in meters of kinect sensor relative to the viewport's center.
    "z": 1.7, # height in meters of actual kinect sensor from the floor.
    "roll": 0, # angle in degrees of sensor's roll (used for INU input - trig function for this is commented out by default).
    "azimuth": 0, # sensor's yaw angle in degrees.
    "elevation": -15, # sensor's pitch angle in degrees.
}

您应根据传感器的实际位置和方向更新这些信息:

两个最重要的参数是theta(仰角)角度和距地面的高度.我只使用了一个简单的卷尺和一个经过校准的眼睛，但是有一天我打算提供编码器或INU数据来实时更新这些参数(随着传感器的移动).

The two most important parameters are the theta (elevation) angle and the height from the floor. A simple measuring tape and a calibrated eye is all I used, however I intend on someday feeding encoder or INU data to update these parameters in real-time (as the sensor is moved around).

同样，所有更改都已反映在功能示例中.

如果有人成功地对此示例进行了改进，或者对使事情变得更紧凑的方法提出了建议，那么如果您可以留下评论来解释细节，我将不胜感激.

以下是此项目的完整功能示例:

#! /usr/bin/python

#--------------------------------#
# Kinect v2 point cloud visualization using a Numpy based 
# real-world coordinate processing algorithm and OpenGL.
#--------------------------------#

import sys
import numpy as np

from pyqtgraph.Qt import QtCore, QtGui
import pyqtgraph.opengl as gl

from pylibfreenect2 import Freenect2, SyncMultiFrameListener
from pylibfreenect2 import FrameType, Registration, Frame, libfreenect2

fn = Freenect2()
num_devices = fn.enumerateDevices()
if num_devices == 0:
    print("No device connected!")
    sys.exit(1)

serial = fn.getDeviceSerialNumber(0)
device = fn.openDevice(serial)

types = 0
types |= FrameType.Color
types |= (FrameType.Ir | FrameType.Depth)
listener = SyncMultiFrameListener(types)

# Register listeners
device.setColorFrameListener(listener)
device.setIrAndDepthFrameListener(listener)

device.start()

# NOTE: must be called after device.start()
registration = Registration(device.getIrCameraParams(),
                            device.getColorCameraParams())

undistorted = Frame(512, 424, 4)
registered = Frame(512, 424, 4)


#QT app
app = QtGui.QApplication([])
gl_widget = gl.GLViewWidget()
gl_widget.show()
gl_grid = gl.GLGridItem()
gl_widget.addItem(gl_grid)

#initialize some points data
pos = np.zeros((1,3))

sp2 = gl.GLScatterPlotItem(pos=pos)
sp2.setGLOptions('opaque') # Ensures not to allow vertexes located behinde other vertexes to be seen.

gl_widget.addItem(sp2)

# Kinects's intrinsic parameters based on v2 hardware (estimated).
CameraParams = {
  "cx":254.878,
  "cy":205.395,
  "fx":365.456,
  "fy":365.456,
  "k1":0.0905474,
  "k2":-0.26819,
  "k3":0.0950862,
  "p1":0.0,
  "p2":0.0,
}

def depthToPointCloudPos(x_d, y_d, z, scale=1000):
    # This runs in Python slowly as it is required to be called from within a loop, but it is a more intuitive example than it's vertorized alternative (Purly for example)
    # calculate the real-world xyz vertex coordinate from the raw depth data (one vertex at a time).
    x = (x_d - CameraParams['cx']) * z / CameraParams['fx']
    y = (y_d - CameraParams['cy']) * z / CameraParams['fy']

    return x / scale, y / scale, z / scale

def depthMatrixToPointCloudPos(z, scale=1000):
    # bacically this is a vectorized version of depthToPointCloudPos()
    # calculate the real-world xyz vertex coordinates from the raw depth data matrix.
    C, R = np.indices(z.shape)

    R = np.subtract(R, CameraParams['cx'])
    R = np.multiply(R, z)
    R = np.divide(R, CameraParams['fx'] * scale)

    C = np.subtract(C, CameraParams['cy'])
    C = np.multiply(C, z)
    C = np.divide(C, CameraParams['fy'] * scale)

    return np.column_stack((z.ravel() / scale, R.ravel(), -C.ravel()))

# Kinect's physical orientation in the real world.
CameraPosition = {
    "x": 0, # actual position in meters of kinect sensor relative to the viewport's center.
    "y": 0, # actual position in meters of kinect sensor relative to the viewport's center.
    "z": 1.7, # height in meters of actual kinect sensor from the floor.
    "roll": 0, # angle in degrees of sensor's roll (used for INU input - trig function for this is commented out by default).
    "azimuth": 0, # sensor's yaw angle in degrees.
    "elevation": -15, # sensor's pitch angle in degrees.
}

def applyCameraOrientation(pt):
    # Kinect Sensor Orientation Compensation
    # This runs slowly in Python as it is required to be called within a loop, but it is a more intuitive example than it's vertorized alternative (Purly for example)
    # use trig to rotate a vertex around a gimbal.
    def rotatePoints(ax1, ax2, deg):
        # math to rotate vertexes around a center point on a plane.
        hyp = np.sqrt(pt[ax1] ** 2 + pt[ax2] ** 2) # Get the length of the hypotenuse of the real-world coordinate from center of rotation, this is the radius!
        d_tan = np.arctan2(pt[ax2], pt[ax1]) # Calculate the vertexes current angle (returns radians that go from -180 to 180)

        cur_angle = np.degrees(d_tan) % 360 # Convert radians to degrees and use modulo to adjust range from 0 to 360.
        new_angle = np.radians((cur_angle + deg) % 360) # The new angle (in radians) of the vertexes after being rotated by the value of deg.

        pt[ax1] = hyp * np.cos(new_angle) # Calculate the rotated coordinate for this axis.
        pt[ax2] = hyp * np.sin(new_angle) # Calculate the rotated coordinate for this axis.

    #rotatePoints(0, 2, CameraPosition['roll']) #rotate on the Y&Z plane # Disabled because most tripods don't roll. If an Inertial Nav Unit is available this could be used)
    rotatePoints(1, 2, CameraPosition['elevation']) #rotate on the X&Z plane
    rotatePoints(0, 1, CameraPosition['azimuth']) #rotate on the X&Y plane

    # Apply offsets for height and linear position of the sensor (from viewport's center)
    pt[:] += np.float_([CameraPosition['x'], CameraPosition['y'], CameraPosition['z']])



    return pt

def applyCameraMatrixOrientation(pt):
    # Kinect Sensor Orientation Compensation
    # bacically this is a vectorized version of applyCameraOrientation()
    # uses same trig to rotate a vertex around a gimbal.
    def rotatePoints(ax1, ax2, deg):
        # math to rotate vertexes around a center point on a plane.
        hyp = np.sqrt(pt[:, ax1] ** 2 + pt[:, ax2] ** 2) # Get the length of the hypotenuse of the real-world coordinate from center of rotation, this is the radius!
        d_tan = np.arctan2(pt[:, ax2], pt[:, ax1]) # Calculate the vertexes current angle (returns radians that go from -180 to 180)

        cur_angle = np.degrees(d_tan) % 360 # Convert radians to degrees and use modulo to adjust range from 0 to 360.
        new_angle = np.radians((cur_angle + deg) % 360) # The new angle (in radians) of the vertexes after being rotated by the value of deg.

        pt[:, ax1] = hyp * np.cos(new_angle) # Calculate the rotated coordinate for this axis.
        pt[:, ax2] = hyp * np.sin(new_angle) # Calculate the rotated coordinate for this axis.

    #rotatePoints(1, 2, CameraPosition['roll']) #rotate on the Y&Z plane # Disabled because most tripods don't roll. If an Inertial Nav Unit is available this could be used)
    rotatePoints(0, 2, CameraPosition['elevation']) #rotate on the X&Z plane
    rotatePoints(0, 1, CameraPosition['azimuth']) #rotate on the X&Y

    # Apply offsets for height and linear position of the sensor (from viewport's center)
    pt[:] += np.float_([CameraPosition['x'], CameraPosition['y'], CameraPosition['z']])



    return pt


def update():
    colors = ((1.0, 1.0, 1.0, 1.0))

    frames = listener.waitForNewFrame()

    # Get the frames from the Kinect sensor
    ir = frames["ir"]
    color = frames["color"]
    depth = frames["depth"]

    d = depth.asarray() #the depth frame as an array (Needed only with non-vectorized functions)

    registration.apply(color, depth, undistorted, registered)

    # Format the color registration map - To become the "color" input for the scatterplot's setData() function.
    colors = registered.asarray(np.uint8)
    colors = np.divide(colors, 255) # values must be between 0.0 - 1.0
    colors = colors.reshape(colors.shape[0] * colors.shape[1], 4 ) # From: Rows X Cols X RGB -to- [[r,g,b],[r,g,b]...]
    colors = colors[:, :3:]  # remove alpha (fourth index) from BGRA to BGR
    colors = colors[...,::-1] #BGR to RGB

    # Calculate a dynamic vertex size based on window dimensions and camera's position - To become the "size" input for the scatterplot's setData() function.
    v_rate = 5.0 # Rate that vertex sizes will increase as zoom level increases (adjust this to any desired value).
    v_scale = np.float32(v_rate) / gl_widget.opts['distance'] # Vertex size increases as the camera is "zoomed" towards center of view.
    v_offset = (gl_widget.geometry().width() / 1000)**2 # Vertex size is offset based on actual width of the viewport.
    v_size = v_scale + v_offset

    # Calculate 3d coordinates (Note: five optional methods are shown - only one should be un-commented at any given time)

    """
    # Method 1 (No Processing) - Format raw depth data to be displayed
    m, n = d.shape
    R, C = np.mgrid[:m, :n]
    out = np.column_stack((d.ravel() / 4500, C.ravel()/m, (-R.ravel()/n)+1))
    """

    # Method 2 (Fastest) - Format and compute the real-world 3d coordinates using a fast vectorized algorithm - To become the "pos" input for the scatterplot's setData() function.
    out = depthMatrixToPointCloudPos(undistorted.asarray(np.float32))

    """
    # Method 3 - Format undistorted depth data to real-world coordinates
    n_rows, n_columns = d.shape
    out = np.zeros((n_rows * n_columns, 3), dtype=np.float32)
    for row in range(n_rows):
        for col in range(n_columns):
            z = undistorted.asarray(np.float32)[row][col]
            X, Y, Z = depthToPointCloudPos(row, col, z)
            out[row * n_columns + col] = np.array([Z, Y, -X])
    """

    """
    # Method 4 - Format undistorted depth data to real-world coordinates
    n_rows, n_columns = d.shape
    out = np.zeros((n_rows * n_columns, 3), dtype=np.float64)
    for row in range(n_rows):
        for col in range(n_columns):
            X, Y, Z = registration.getPointXYZ(undistorted, row, col)
            out[row * n_columns + col] = np.array([Z, X, -Y])
    """

    """
    # Method 5 - Format undistorted and regisered data to real-world coordinates with mapped colors (dont forget color=colors in setData)
    n_rows, n_columns = d.shape
    out = np.zeros((n_rows * n_columns, 3), dtype=np.float64)
    colors = np.zeros((d.shape[0] * d.shape[1], 3), dtype=np.float64)
    for row in range(n_rows):
        for col in range(n_columns):
            X, Y, Z, B, G, R = registration.getPointXYZRGB(undistorted, registered, row, col)
            out[row * n_columns + col] = np.array([Z, X, -Y])
            colors[row * n_columns + col] = np.divide([R, G, B], 255)
    """


    # Kinect sensor real-world orientation compensation.
    out = applyCameraMatrixOrientation(out)

    """
    # For demonstrating the non-vectorized orientation compensation function (slow)
    for i, pt in enumerate(out):
        out[i] = applyCameraOrientation(pt)
    """


    # Show the data in a scatter plot
    sp2.setData(pos=out, color=colors, size=v_size)

    # Lastly, release frames from memory.
    listener.release(frames)

t = QtCore.QTimer()
t.timeout.connect(update)
t.start(50)


## Start Qt event loop unless running in interactive mode.
if __name__ == '__main__':
    import sys
    if (sys.flags.interactive != 1) or not hasattr(QtCore, 'PYQT_VERSION'):
        QtGui.QApplication.instance().exec_()

device.stop()
device.close()

sys.exit(0)

推荐答案

这并不是一个完整的答案...我只是想指出，您正在创建很多临时数组，可以在其中进行操作更多的就地操作:

This isn't intended to be a complete answer... I just wanted to point out that you are creating a lot of temporary arrays, where you could do more of the operations in-place:

def depthMatrixToPointCloudPos2(z, scale=1000):

    R, C = numpy.indices(z.shape)

    R -= CameraParams['cx'])
    R *= z
    R /= CameraParams['fx'] * scale

    C -= CameraParams['cy']
    C *= z
    C /= CameraParams['fy'] * scale

    return np.column_stack((z.ravel() / scale, R.ravel(), -C.ravel()))

(如果我正确阅读了您的代码.)

(If I read your code correctly.)

此外，请注意数据类型，如果您使用的是64位计算机，则默认情况下将为64位.您能否摆脱较小的类型，以减少要处理的数据量?

Also, note the data types, which if you are on a 64-bit machine, will be 64-bit by default. Can you get away with smaller types, to cut down on the amount of data to crunch?

这篇关于向量化Kinect现实世界中的坐标处理算法以提高速度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！