深度和颜色框架对齐 [英] Depth and color frames alignment

问题描述

似乎这个问题已经被问了几次，但是尽管阅读了所有其他的线程和文档，我仍然无法找到一个可行的解决方案。

It seems that this question has been asked a few times already, but despite reading all the other threads and documentation I still can't come up with a working solution.

我有一个程序，允许将单个帧保存为图片。当我保存深度和颜色帧时，由于视差失真，它们显然未对准。我的目标是让颜色框架与深度框架对齐，以便生成的图像在放在彼此顶部时完全对齐
。

I have a program which allows saving individual frames as pictures. When I save depth and color frames they are obviously misaligned due to parallax distortion. My goal is to have the color frame aligned to the depth frame so that produced images align perfectly when placed on top of each other.

我尝试过几种不同的东西但是似乎没什么用，因为我不完全了解如何使用INuiCoordinateMapper。这是我的对齐方法的最新版本（depthData和colorData是从NUI_LOCKED_RECT.pBits获得的，mappedData是
输出）：

I've tried several different things but nothing seems to work as I don't fully understand how to use INuiCoordinateMapper. Here's a recent version of my alignment method (depthData and colorData are obtained from NUI_LOCKED_RECT.pBits, mappedData is the output):

bool mapColorFrameToDepthFrame(unsigned char *depthData, unsigned char* colorData, unsigned char* mappedData)
{
    INuiCoordinateMapper* coordMapper;

    // Get coordinate mapper
    m_pSensor->NuiGetCoordinateMapper(&coordMapper);

    NUI_DEPTH_IMAGE_POINT* depthPoints = new NUI_DEPTH_IMAGE_POINT[640 * 480];

    HRESULT result = coordMapper->MapColorFrameToDepthFrame(NUI_IMAGE_TYPE_COLOR, NUI_IMAGE_RESOLUTION_640x480, NUI_IMAGE_RESOLUTION_640x480, 640 * 480, reinterpret_cast<NUI_DEPTH_IMAGE_PIXEL*>(depthData), 640 * 480, depthPoints);
    if (FAILED(result))
    {
        return false;
    }    

    int pos = 0;
    int* colorRun = reinterpret_cast<int*>(colorData);
    int* mappedRun = reinterpret_cast<int*>(mappedData);

    // For each pixel of new color frame
    for (int i = 0; i < 640 * 480; ++i)
    {
        // Find the corresponding pixel in original color frame from depthPoints
        pos = (depthPoints[i].y * 640) + depthPoints[i].x;

        // Set pixel value if it's within frame boundaries
        if (pos < 640 * 480)
        {
            mappedRun[i] = colorRun[pos];
        }
    }

    return true;
}

我的结果看起来很随意（生成的颜色框架大部分是灰色的，图像的随机部分在这里和那里） ，所以我想我可能都错了。

My results seem pretty random (the resulting color frame is mostly gray with random pieces of the image here and there), so I assume I may have it all wrong.

推荐答案

你传递的数据是deepData ？您需要确保它是NUI_DEPTH_IMAGE_PIXEL格式的数据。以下是获取它的调用顺序：

What data are you passing as depthData? You need to be sure that it's the data in the NUI_DEPTH_IMAGE_PIXEL format. Here's the sequence of calls to get it:

    HRESULT hr;
    NUI_IMAGE_FRAME imageFrame;

    // Attempt to get the depth frame
    hr = m_pNuiSensor->NuiImageStreamGetNextFrame(m_pDepthStreamHandle, 0, &imageFrame);
    if (SUCCEEDED(hr))
    {
        BOOL nearMode;
        INuiFrameTexture* pTexture;

        // Get the depth image pixel texture
        hr = m_pNuiSensor->NuiImageFrameGetDepthImagePixelFrameTexture(
            m_pDepthStreamHandle, &imageFrame, &nearMode, &pTexture);
        if (SUCCEEDED(hr))
        {

            NUI_LOCKED_RECT LockedRect;

            // Lock the frame data so the Kinect knows not to modify it while we're reading it
            pTexture->LockRect(0, &LockedRect, NULL, 0);

            // Make sure we've received valid data
            if (LockedRect.Pitch != 0)
            {
                depthPixels = LockedRect.pBits;
            }

            // We're done with the texture so unlock it
            pTexture->UnlockRect(0);

            pTexture->Release();
        }

        // Release the frame
        m_pNuiSensor->NuiImageStreamReleaseFrame(m_pDepthStreamHandle, &imageFrame);
    }

这篇关于深度和颜色框架对齐的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！