Kinect背景移除 [英] Kinect background removal
问题描述
I followed the code provided by Robert Levy at this link: http://channel9.msdn.com/coding4fun/kinect/Display-Kinect-color-image-containing-only-players-aka-background-removal
我尝试将其实现到现有代码中，但结果不一致.如果用户在程序启动时位于kinect的视野中，则有时会删除背景.如果用户走进视野，它将不会接住他们.
I tried implementing it into my existing code, and have had inconsistent results. If the user is in the kinect's field of view when the program starts up it will remove the background some of the time. If the user walks into the field of view it will not pick them up.
namespace KinectUserRecognition
{
public partial class MainWindow : Window
{
public MainWindow()
{
InitializeComponent();
}
//Kinect Runtime
Runtime kinect = Runtime.Kinects[0];
PlanarImage colorImage;
PlanarImage depthImage;
bool isDepthImage;
WriteableBitmap player1;
private void Window_Loaded(object sender, RoutedEventArgs e)
{
isDepthImage = false;
//UseDepthAndPlayerIndex and UseSkeletalTracking
kinect.Initialize(RuntimeOptions.UseDepthAndPlayerIndex | RuntimeOptions.UseColor);// | RuntimeOptions.UseSkeletalTracking);
//register for event
kinect.VideoFrameReady += new EventHandler<ImageFrameReadyEventArgs>(nui_VideoFrameReady);
kinect.DepthFrameReady += new EventHandler<ImageFrameReadyEventArgs>(nui_DepthFrameReady);
//Video image type
kinect.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480,
ImageType.Color);
//DepthAndPlayerIndex ImageType
kinect.DepthStream.Open(ImageStreamType.Depth, 2, ImageResolution.Resolution320x240,
ImageType.DepthAndPlayerIndex);
}
void nui_VideoFrameReady(object sender, ImageFrameReadyEventArgs e)
{
colorImage = e.ImageFrame.Image;
image1.Source = BitmapSource.Create(colorImage.Width, colorImage.Height, 96, 96,
PixelFormats.Bgr32, null, colorImage.Bits, colorImage.Width * colorImage.BytesPerPixel);
if (isDepthImage)
{
player1 = GeneratePlayerImage(e.ImageFrame, 1);
image3.Source = player1;
}
}
void nui_DepthFrameReady(object sender, ImageFrameReadyEventArgs e)
{
//Convert depth information for a pixel into color information
byte[] ColoredBytes = GenerateColoredBytes(e.ImageFrame);
depthImage = e.ImageFrame.Image;
image2.Source = BitmapSource.Create(depthImage.Width, depthImage.Height, 96, 96, PixelFormats.Bgr32, null,
ColoredBytes, depthImage.Width * PixelFormats.Bgr32.BitsPerPixel / 8);
isDepthImage = true;
}
private WriteableBitmap GeneratePlayerImage(ImageFrame imageFrame, int playerIndex)
{
int depthWidth = kinect.DepthStream.Width;
int depthHeight = kinect.DepthStream.Height;
WriteableBitmap target = new WriteableBitmap(depthWidth, depthHeight, 96, 96, PixelFormats.Bgra32, null);
var depthRect = new System.Windows.Int32Rect(0, 0, depthWidth, depthHeight);
byte[] color = imageFrame.Image.Bits;
byte[] output = new byte[depthWidth * depthHeight * 4];
//loop over each pixel in the depth image
int outputIndex = 0;
for (int depthY = 0, depthIndex = 0; depthY < depthHeight; depthY++)
{
for(int depthX = 0; depthX < depthWidth; depthX++, depthIndex +=2)
{
short depthValue = (short)(depthImage.Bits[depthIndex] | (depthImage.Bits[depthIndex + 1] << 8));
int colorX, colorY;
kinect.NuiCamera.GetColorPixelCoordinatesFromDepthPixel(
imageFrame.Resolution,
imageFrame.ViewArea,
depthX, depthY, //depth coordinate
depthValue, //depth value
out colorX, out colorY); //color coordinate
//ensure that the calculate color location is within the bounds of the image
colorX = Math.Max(0, Math.Min(colorX, imageFrame.Image.Width - 1));
colorY = Math.Max(0, Math.Min(colorY, imageFrame.Image.Height - 1));
output[outputIndex++] = color[(4 * (colorX + (colorY * imageFrame.Image.Width))) + 0];
output[outputIndex++] = color[(4 * (colorX + (colorY * imageFrame.Image.Width))) + 1];
output[outputIndex++] = color[(4 * (colorX + (colorY * imageFrame.Image.Width))) + 2];
output[outputIndex++] = GetPlayerIndex(depthImage.Bits[depthIndex]) == playerIndex ? (byte)255 : (byte)0;
}
}
target.WritePixels(depthRect, output, depthWidth * PixelFormats.Bgra32.BitsPerPixel / 8, 0);
return target;
//return output;
}
private static int GetPlayerIndex(byte firstFrame)
{
//returns 0 = no player, 1 = 1st player, 2 = 2nd player...
//bitwise & on firstFrame
return (int)firstFrame & 7;
}
}
}
-编辑1-
我认为我已经缩小了问题的范围，但是不确定如何解决这个问题.我假设在kinect的视野中只有一个人会从我的"GetPlayerIndex"方法返回一个值.不是这种情况.我希望为每个删除背景的人制作一张单独的图像.我应该假定从哪种类型的值中接收:
I think I've narrowed the problem down, but I'm not sure of a way to resolve it. I assumed that having only one person in the kinect's field of view would return a value of one from my "GetPlayerIndex" method. This is not the case. I was hoping to produce a separate image for each person with the background removed. What type of values should I assume to receive from:
-编辑2-
在测试中，我注意到玩家索引的最大值为6，但是获得的索引不一致.是否有办法知道将哪个玩家索引分配给骨骼?例如，如果我是fov中唯一的人，是否有办法知道我的玩家指数始终为1?
From my tests I've noticed that I can a max value of 6 for the player index, but the index that I get isn't consistent. If there a way to know what player index will be assigned to a skeleton? For example, if I were the only person in the fov would there be a way to know that my player index would always be 1?
推荐答案
不能保证玩家索引是任何东西.一旦抓住骨架，索引将一直保持不变，直到它消失为止，但是您不能假定第一个玩家将是1，第二个玩家是2，依此类推.
The player index is not guaranteed to be anything. Once it catches a skeleton, the index will stay the same for that skeleton until it loses sight of it, but you can't assume that the first player will be 1, the second 2, etc.
您需要做的是在player1 = GeneratePlayerImage(e.ImageFrame, 1);调用之前确定有效的骨架索引，或者更改GeneratePlayerImage函数以找到索引.如果您只想移除背景并保持画面中所有人物的像素不变，只需更改以下内容即可:
What you'll need to do is determine a valid skeleton index prior to the player1 = GeneratePlayerImage(e.ImageFrame, 1); call, or alter the GeneratePlayerImage function to find an index. If you're only interested in removing the background and leaving the pixels for all the people in the frame untouched, just change this:
output[outputIndex++] = GetPlayerIndex(depthImage.Bits[depthIndex]) == playerIndex ? (byte)255 : (byte)0;
对此，它将只检查ANY玩家，而不是特定的玩家:
to this, which will just check for ANY player, instead of a specific player:
output[outputIndex++] = GetPlayerIndex(depthImage.Bits[depthIndex]) != 0 ? (byte)255 : (byte)0;
我可以想到的另外两种方式是针对特定玩家而不是所有玩家:
The other two ways I can think of to do this for a specific player instead of all players:
- 打开Kinect的骨骼供稿，并遍历骨骼数组，它可以为您找到有效的索引.创建一个全局整数来保存该索引，然后使用该全局整数调用GeneratePlayerImage方法.
- 更改GeneratePlayerImage方法以检查每个像素中的播放器索引，如果找到了该索引，请使用该索引删除整个图像的背景(忽略它找到的任何其他索引).
这篇关于Kinect背景移除的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
