使用Matlab录制/播放Kinect-上下颠倒并以黑白显示，而不是灰度 [英] Recording/playback Kinect with Matlab - upside down and in black/white rather than gray scale
问题描述
我正在尝试编写一个程序来记录和回放来自xbox kinect的颜色流和深度流，以简化对图像处理程序的测试.目前，我已经完成了大部分工作，并且颜色流工作正常.但是我在深度流方面遇到了麻烦.
I am trying to write a program to record and playback both colour and depth streams from a xbox kinect to ease testing image processing programs. Currently I have the bulk done and the colour stream works fine. I am however having trouble with the depth stream.
当前，深度流正在上下颠倒播放，并且仅以黑白显示.对于为什么会这样，我有3个想法: 1)从11位转换为8位 2)Motion JPEG 2000格式(以前从未使用过) 3)颜色图错误
Currently, the depth stream is playing back upside down and only in black and white. I have 3 thoughts as to why this may be the case: 1) The conversion to 8-bit from 11-bit 2) the Motion JPEG 2000 format (never used this before) 3) the colormap is wrong
下面是我正在使用的代码.我觉得我不是唯一一个尝试这样做的人，所以我将不胜感激，因为我无法在网络上找到任何具体的东西.
Below is the code I am using. I feel I cannot be the only one trying to do this so any pointer and help would be much appreciated as I couldn't find anything this specific on the web.
%------------------------------------------------
%------------------------------------------------
%Code to record kinect colour and sensor data
%using code supplied on http://www.mathworks.co.uk/help/imaq/examples/using-the- kinect-r-for-windows-r-from-image-acquisition-toolbox-tm.html
%and http://www.mathworks.co.uk/help/imaq/examples/logging-data-to-disk.html
%------------------------------------------------
%------------------------------------------------
dbstop if error
imaqreset %deletes any image acquisition objects that exsist in memory and uploads all adaptors loaded by the toolbox. As a result, image acquisition hardware is reset
%------------------------------------------------
%setting up video streams
%------------------------------------------------
disp('Setting up video streams');
%Call up dicertory containing utility functions
utilpath = fullfile(matlabroot, 'toolbox', 'imaq', 'imaqdemos', 'html', 'KinectForWindows');
addpath(utilpath);
%Create the videoinput objects for the colour and depth streams
colourVid = videoinput('kinect', 1, 'RGB_640x480');
%preview(colourVid);
depthVid = videoinput('kinect', 2, 'Depth_640x480');
%set backlight compensation with centre priority
%set(colourVid, 'BacklightCompensation', 'CentrePriority');
%Set camera angle to 0
%set(colourVid, 'CameraElevationAngle', 0);
disp('Video stream set-up complete');
%------------------------------------------------
%setting up record
%------------------------------------------------
% set the data streams to logging mode and to disk
set(colourVid, 'LoggingMode', 'Disk&Memory');
set(depthVid, 'LoggingMode', 'Disk&Memory');
%Set a video timeout property limit to 50 seconds from
%www.mathworks.com/matlabcentral/answers/103543-why-do-i-receive-the-error-getdata-timed-out-before-frames-were-available-when-using-getdata-in-im
set(colourVid, 'Timeout',50);
set(depthVid, 'Timeout',50);
%Creat a VideoReader object
colourLogfile = VideoWriter('colourTrial5.mj2', 'Motion JPEG 2000');
depthLogfile = VideoWriter('depthTrial5.mj2', 'Motion JPEG 2000');
%configure the video input object to use the VideoWriter object
colourVid.DiskLogger = colourLogfile;
depthVid.DiskLogger = depthLogfile;
%set the triggering mode to 'manual'
triggerconfig([colourVid depthVid], 'manual');
%set the FramePerTrigger property of the VIDEOINPUT objects to 100 to
%acquire 100 frames per trigger.
set([colourVid depthVid], 'FramesPerTrigger', 200);
disp('Video record set-up complete');
%------------------------------------------------
%Initiating the aquisition
%------------------------------------------------
disp('Starting Steam');
%Start the colour and depth device. This begins acquisition, but does not
%start logging of acquired data
start([colourVid depthVid]);
pause(20); %allow time for both streams to start
%Trigger the devices to start logging of data.
trigger([colourVid depthVid]);
%Retrieve the acquired data
[colourFrameData, colourTimeData, colourMetaData] = getdata(colourVid);
[depthFrameData, depthTimeData, depthMetaData] = getdata(depthVid);
stop([colourVid depthVid])
disp('Recording Complete')
%------------------------------------------------
%Play back recordings
%------------------------------------------------
disp('Construct playback objects')
colourPlayback = VideoReader('colourTrial5.mj2');
depthPlayback = VideoReader('depthTrial5.mj2');
%Set colour(c) playback parameters
cFrames = colourPlayback.NumberOfFrames;
cHeight = colourPlayback.Height;
cWidth = colourPlayback.Width;
%Preallocate movie structure
colourMov(1:cFrames)=struct('cdata', zeros(cHeight,cWidth,3,'uint8'),'colormap',[]);
disp('Reading colour frames one by one')
%read one frame at a time
for k = 1:cFrames
colourMov(k).cdata=read(colourPlayback,k);
end
disp('Sizing figure for colour playback')
%Size a figure based on the video's width and height
hf1=figure;
set(hf1,'position',[150 150 cWidth cHeight])
disp('Playing Colour recording')
%play back the movie once at the video's frame rate
movie(hf1,colourMov,1,colourPlayback.FrameRate);
%Set depth(d) playback parameters
dFrames = depthPlayback.NumberOfFrames;
dHeight = depthPlayback.Height;
dWidth = depthPlayback.Width;
%Preallocate movie structure
depthMov(1:dFrames)=struct('cdata', zeros(dHeight,dWidth,3,'uint8'),'colormap',gray(256));
disp('Reading depth frames one by one')
%read one frame at a time
for k = 1:dFrames
depthMov(k).cdata=uint8(read(depthPlayback,k));
%depthMov(k)=imrotate(depthMov(k),180); %tried this to no effect
end
disp('Sizing figure for depth playback')
%Size a figure based on the video's width and height
hf2=figure;
set(hf2,'position',[150 150 dWidth dHeight])
disp('Playing Depth recording')
%play back the movie once at the video's frame rate
movie(hf2,depthMov,1,depthPlayback.FrameRate);
%clear videos from workspace
delete([colourVid depthVid])
clear [colourVid depthVid]
推荐答案
您的脚本编写正确，正确无误，除了如何显示深度数据.
Your script is well-written and correct, except for how you display the depth data.
Kinect将深度记录为16位无符号整数.深度框中的像素值是该像素中任何对象与相机平面的距离(以毫米为单位).因此，如果您关心的某个对象的像素的深度值显示为例如1723，则意味着该对象的一部分距离相机有1.723米.
The Kinect records depth as 16-bit unsigned integers. The pixel value in the depth frame is the distance in millimeters of whatever object is in that pixel from the plane of the camera. So, if the depth value of a pixel of an object you care about reads out as, e.g., 1723, that means that part of the object is 1.723 meters away from the camera.
那么，这与坏掉的显示功能有什么关系?这是您的旧显示代码段:
So, how does that relate to your broken display functionality? Here's your old display snippet:
%read one frame at a time
for k = 1:dFrames
depthMov(k).cdata=uint8(read(depthPlayback,k));
end
问题就在uint8处.我确定您的许多深度值都超过255，这意味着场景中的对象距离相机的距离超过0.2米.实际上，Kinect甚至无法感知到接近的数据！ 您可以检测到的最接近深度，将DepthMode设置为Near，约为0.4米.
The problem is right there at uint8. Many of your depth values I'm sure are beyond 255, meaning objects in your scene are farther than 0.2 meters from the camera. In fact, the Kinect cannot even sense data that close! The nearest you can detect depth, with DepthMode set to Near, is about 0.4 meters.
因此，以下是解决显示问题的方法:
So, here's how to solve the display issue:
%read one frame at a time
maxDistFromCamera = 1600; % 1600 millimeters
for k = 1:dFrames
% Depth frames are int16.
depthFrame = read(depthPlayback,k);
% We'll rescale the image from [0,maxDistFromCamera] to [0,255]
depthFrame = 255.0*single(depthFrame)/maxDistFromCamera;
% And then recast it to uint8 for display.
depthMov(k).cdata=uint8(depthFrame);
end
为方便起见，下面复制了整个脚本以及我的编辑内容.
Below is reproduced the whole script, with my edits, for convenience.
%------------------------------------------------
%------------------------------------------------
%Code to record kinect colour and sensor data
%using code supplied on http://www.mathworks.co.uk/help/imaq/examples/using-the- kinect-r-for-windows-r-from-image-acquisition-toolbox-tm.html
%and http://www.mathworks.co.uk/help/imaq/examples/logging-data-to-disk.html
%------------------------------------------------
%------------------------------------------------
imaqreset %deletes any image acquisition objects that exsist in memory and uploads all adaptors loaded by the toolbox. As a result, image acquisition hardware is reset
%------------------------------------------------
%setting up video streams
%------------------------------------------------
disp('Setting up video streams');
%Call up dicertory containing utility functions
utilpath = fullfile(matlabroot, 'toolbox', 'imaq', 'imaqdemos', 'html', 'KinectForWindows');
addpath(utilpath);
%Create the videoinput objects for the colour and depth streams
colourVid = videoinput('kinect', 1, 'RGB_640x480');
%preview(colourVid);
depthVid = videoinput('kinect', 2, 'Depth_320x240');
% Set the depth mode to near.
srcDepth = getselectedsource(depthVid);
srcColor = getselectedsource(colourVid);
set(srcDepth, 'DepthMode' , 'Near');
set(srcDepth, 'CameraElevationAngle', 0);
%set backlight compensation with centre priority
set(srcColor, 'BacklightCompensation', 'CenterPriority');
disp('Video stream set-up complete');
%------------------------------------------------
%setting up record
%------------------------------------------------
% set the data streams to logging mode and to disk
set(colourVid, 'LoggingMode', 'Disk&Memory');
set(depthVid, 'LoggingMode', 'Disk&Memory');
%Set a video timeout property limit to 50 seconds from
%www.mathworks.com/matlabcentral/answers/103543-why-do-i-receive-the-error-getdata-timed-out-before-frames-were-available-when-using-getdata-in-im
set(colourVid, 'Timeout',50);
set(depthVid, 'Timeout',50);
%Creat a VideoReader object
colourLogfile = VideoWriter('colourTrial5.mj2', 'Motion JPEG 2000');
depthLogfile = VideoWriter('depthTrial5.mj2', 'Archival');
%configure the video input object to use the VideoWriter object
colourVid.DiskLogger = colourLogfile;
depthVid.DiskLogger = depthLogfile;
%set the triggering mode to 'manual'
triggerconfig([colourVid depthVid], 'manual');
%set the FramePerTrigger property of the VIDEOINPUT objects to 100 to
%acquire 100 frames per trigger.
set([colourVid depthVid], 'FramesPerTrigger', 30);
disp('Video record set-up complete');
%------------------------------------------------
%Initiating the aquisition
%------------------------------------------------
disp('Starting Steam');
%Start the colour and depth device. This begins acquisition, but does not
%start logging of acquired data
start([colourVid depthVid]);
pause(20); %allow time for both streams to start
disp('Starting Depth Stream');
%Trigger the devices to start logging of data.
trigger([colourVid depthVid]);
%Retrieve the acquired data
[colourFrameData, colourTimeData, colourMetaData] = getdata(colourVid);
[depthFrameData, depthTimeData, depthMetaData] = getdata(depthVid);
stop([colourVid depthVid])
disp('Recording Complete')
%------------------------------------------------
%Play back recordings
%------------------------------------------------
disp('Construct playback objects')
colourPlayback = VideoReader('colourTrial5.mj2');
depthPlayback = VideoReader('depthTrial5.mj2');
%Set colour(c) playback parameters
cFrames = colourPlayback.NumberOfFrames;
cHeight = colourPlayback.Height;
cWidth = colourPlayback.Width;
%Preallocate movie structure
colourMov(1:cFrames)=struct('cdata', zeros(cHeight,cWidth,3,'uint8'),'colormap',[]);
disp('Reading colour frames one by one')
%read one frame at a time
for k = 1:cFrames
colourMov(k).cdata=read(colourPlayback,k);
end
disp('Sizing figure for colour playback')
%Size a figure based on the video's width and height
hf1=figure;
set(hf1,'position',[150 150 cWidth cHeight])
disp('Playing Colour recording')
%play back the movie once at the video's frame rate
movie(hf1,colourMov,1,colourPlayback.FrameRate);
%Set depth(d) playback parameters
dFrames = depthPlayback.NumberOfFrames;
dHeight = depthPlayback.Height;
dWidth = depthPlayback.Width;
%Preallocate movie structure
depthMov(1:dFrames)=struct('cdata', zeros(dHeight,dWidth,3,'uint8'),'colormap',gray(256));
disp('Reading depth frames one by one')
%read one frame at a time
maxDistFromCamera = 1600; % 1600 millimeters
for k = 1:dFrames
% Depth frames are int16.
depthFrame = read(depthPlayback,k);
% We'll rescale the image from [0,maxDistFromCamera] to [0,255]
depthFrame = 255.0*single(depthFrame)/maxDistFromCamera;
% And then recast it to uint8 for display.
depthMov(k).cdata=uint8(depthFrame);
end
disp('Sizing figure for depth playback')
%Size a figure based on the video's width and height
hf2=figure;
set(hf2,'position',[150 150 dWidth dHeight])
disp('Playing Depth recording')
%play back the movie once at the video's frame rate
movie(hf2,depthMov,1,depthPlayback.FrameRate);
%clear videos from workspace
delete([colourVid depthVid])
clear [colourVid depthVid]
close all;
文本上有一些细微的差异，例如，我的Kinect SDK需要某些参数的拼写不同，而我只是固定内联了.我还禁用了您在顶部的积极调试设置.不重要，仅记录一下即可.最后，我将深度流的 VideoWriter格式设置为Archival ，这样就不会在保存时破坏任何深度值(对于精确的深度测量应用来说，这很重要，这是我最近的工作重点).
There are some minor textual differences, for instance, my Kinect SDK requires different spelling of some parameters, which I just fixed inline. I also disabled the aggressive debugging setting you had on at the top. Not important, just making a note of it. Lastly, I set the VideoWriter format to Archival for the depth stream, so that none of the depth values are corrupted on save (important for precise depth measurement applications, which is what I tend to be up to these days).
这篇关于使用Matlab录制/播放Kinect-上下颠倒并以黑白显示，而不是灰度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
