Tango相机预览RGBIR [英] Tango Camera Preview for RGBIR

问题描述

我使用Tango的demoOverlaySample演示。

我想要看到IR数据（单独或与颜色）而不是颜色。
所以，我在它出现的两个地方替换了TANGO_CAMERA_COLOR和TANGO_CAMERA_RGBIR。

但是屏幕是黑色的。

以下是代码：

  / * 
 *版权所有2014 Google Inc.保留所有权利。 
 * 
 *根据Apache许可证2.0版（许可证）授权; 
 *您不得使用此文件，除非符合许可证。 
 *您可以通过以下方式获取许可证的副本：
 * 
 * http://www.apache.org/licenses/LICENSE-2.0 
 * 
 *除非适用法律要求或以书面形式同意，软件
 *根据许可证分发的软件按原样基础，
 *分发，不附有任何明示或暗示的担保或条件。 
 *请参阅许可证管理权限的特定语言和
 *许可证下的限制。 
 * / 
 package com.projecttango.experiments.videooverlaysample; 
 
 import java.util.ArrayList; 
 
 import android.app.Activity; 
 import android.content.Intent; 
 import android.os.Bundle; 
 import Android.util.Log; 
 import android.widget.Toast; 
 
 import com.google.atap.tangoservice.Tango; 
 import com.google.atap.tangoservice.Tango.OnTangoUpdateListener; 
 import com.google.atap.tangoservice.TangoCameraIntrinsics; 
 import com.google.atap.tangoservice.TangoCameraPreview; 
 import com.google.atap.tangoservice.TangoConfig; 
 import com.google.atap.tangoservice.TangoCoordinateFramePair; 
 import com.google.atap.tangoservice.TangoEvent; 
 import com.google.atap.tangoservice.TangoPoseData; 
 import com.google.atap.tangoservice.TangoXyzIjData; 
 
 / ** 
 *一个示例显示了TangoCameraPreview类的用法
 * TangoCameraPreviewClass的用法：
 *要使用此类，我们首先需要初始化TangoCameraPreview类与活动的
 *上下文，并通过使用connectToTangoCamera类连接到我们想要的摄像头。一旦连接
 *建立，我们需要手动更新TangoCameraPreview的纹理通过使用
 * onFrameAvailable回调。 
 *注意：
 *要使用TangoCameraPreview类，我们需要在最低级别请求用户权限MotionTracking 
 *。这是因为在Java中，所有的回调，例如
 * onPoseAvailable，onXyzIjAvailable，onTangoEvents，onFrameAvailable都设置在一起。 
 * / 
 public class MainActivity extends Activity {
 private TangoCameraPreview tangoCameraPreview; 
 private Tango mTango; 
 private boolean mIsConnected; 
 private boolean mIsPermissionGranted; 
 
 @Override 
 protected void onCreate（Bundle savedInstanceState）{
 super.onCreate（savedInstanceState）; 
 tangoCameraPreview = new TangoCameraPreview（this）; 
 mTango = new Tango（this）; 
 startActivityForResult（
 Tango.getRequestPermissionIntent（Tango.PERMISSIONTYPE_MOTION_TRACKING），
 Tango.TANGO_INTENT_ACTIVITYCODE）; 
 setContentView（tangoCameraPreview）; 
} 
 
 @Override 
 protected void onActivityResult（int requestCode，int resultCode，Intent data）{
 //检查我们响应的请求
 if（requestCode == Tango.TANGO_INTENT_ACTIVITYCODE）{
 //确保请求成功
 if（resultCode == RESULT_CANCELED）{
 Toast.makeText（this，Motion Tracking Permissions Required ！，
 Toast.LENGTH_SHORT）.show（）; 
 finish（）; 
} else {
 startCameraPreview（）; 
 mIsPermissionGranted = true; 
} 
} 
} 
 
 //相机预览
 private void startCameraPreview（）{
 //连接彩色相机
 tangoCameraPreview.connectToTangoCamera（mTango，
 TangoCameraIntrinsics.TANGO_CAMERA_RGBIR）; 
 //为Tango服务使用默认配置。 
 TangoConfig config = mTango.getConfig（TangoConfig.CONFIG_TYPE_DEFAULT）; 
 config.putBoolean（TangoConfig.KEY_BOOLEAN_DEPTH，true）; 
 mTango.connect（config）; 
 mIsConnected = true; 
 
 //不需要添加任何坐标框架对，因为我们不使用
 //姿势数据。所以只是初始化。 
 ArrayList< TangoCoordinateFramePair> framePairs = new ArrayList< TangoCoordinateFramePair>（）; 
 mTango.connectListener（framePairs，new OnTangoUpdateListener（）{
 @Override 
 public void onPoseAvailable（TangoPoseData pose）{
 //我们没有为此应用程序使用OnPoseAvailable 
} 
 
 @Override 
 public void onFrameAvailable（int cameraId）{
 
 //检查可用的帧是否用于我们想要的摄像机，
 //在相机预览上更新其帧。
 if（cameraId == TangoCameraIntrinsics.TANGO_CAMERA_RGBIR）{
 tangoCameraPreview.onFrameAvailable（）; 
} 
} 
 
 @Override 
 public void onXyzIjAvailable（TangoXyzIjData xyzIj）{
 //我们没有为此应用程序使用OnPoseAvailable 
} 
 
 @Override 
 public void onTangEvent（TangoEvent event）{
 //我们没有为此应用程序使用OnPoseAvailable 
} 
}）; 
} 
 
 @Override 
 protected void onPause（）{
 super.onPause（）; 
 if（mIsConnected）{
 mTango.disconnect（）; 
 tangoCameraPreview.disconnectFromTangoCamera（）; 
 mIsConnected = false; 
} 
} 
 
 @Override 
 protected void onResume（）{
 super.onResume（）; 
 if（！mIsConnected&& mIsPermissionGranted）{
 startCameraPreview（）; 
} 
} 
} 
  

解决方案>

您不能使用

  tangoCameraPreview.connectToTangoCamera（mTango，TangoCameraIntrinsics.TANGO_CAMERA_RGBIR）; 
 

Java API不提供与RGBIR，仅Color和Fisheye相机的connectToTangoCamera href =https://developers.google.com/project-tango/apis/java/reference/TangoCameraPreview =nofollow>这里）

要显示深度图像而不是彩色图像，您必须手动计算深度图像。因此，您必须：

1. 将指定时间戳处的点云转换为相机帧。


2. 投影点云点（x，y，z）接收像素（x，y）




4. 将z值存储在其给定的像素位置处。
将相机框架（1240x720） （因为深度相机只有320x180的分辨率）

5. 在OpenGL中，您可以轻松地将该数组用于纹理。






有关详细信息，我建议您查看C示例 rgb-depth-sync




I am using Tango's demo for videoOverlaySample.

Instead of color, I would like to see the IR data (alone or with color). So, I replaced TANGO_CAMERA_COLOR with TANGO_CAMERA_RGBIR in both places where it appears.

But screen is black.

Here is the code:

/*
 * Copyright 2014 Google Inc. All Rights Reserved.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package com.projecttango.experiments.videooverlaysample;

import java.util.ArrayList;

import android.app.Activity;
import android.content.Intent;
import android.os.Bundle;
import android.util.Log;
import android.widget.Toast;

import com.google.atap.tangoservice.Tango;
import com.google.atap.tangoservice.Tango.OnTangoUpdateListener;
import com.google.atap.tangoservice.TangoCameraIntrinsics;
import com.google.atap.tangoservice.TangoCameraPreview;
import com.google.atap.tangoservice.TangoConfig;
import com.google.atap.tangoservice.TangoCoordinateFramePair;
import com.google.atap.tangoservice.TangoEvent;
import com.google.atap.tangoservice.TangoPoseData;
import com.google.atap.tangoservice.TangoXyzIjData;

/**
 * An example showing the usage of TangoCameraPreview class
 * Usage of TangoCameraPreviewClass:
 * To use this class, we first need initialize the TangoCameraPreview class with the activity's 
 * context and connect to the camera we want by using connectToTangoCamera class.Once the connection 
 * is established we need to manually update the TangoCameraPreview's texture by using the
 * onFrameAvailable callbacks.
 * Note:
 * To use TangoCameraPreview class we need to ask the user permissions for MotionTracking 
 * at the minimum level. This is because in Java all the call backs such as 
 * onPoseAvailable,onXyzIjAvailable, onTangoEvents, onFrameAvailable are set together at once. 
 */
public class MainActivity extends Activity {
    private TangoCameraPreview tangoCameraPreview;
    private Tango mTango;
    private boolean mIsConnected;
    private boolean mIsPermissionGranted;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        tangoCameraPreview = new TangoCameraPreview(this);
        mTango = new Tango(this);
        startActivityForResult(
                Tango.getRequestPermissionIntent(Tango.PERMISSIONTYPE_MOTION_TRACKING),
                Tango.TANGO_INTENT_ACTIVITYCODE);
        setContentView(tangoCameraPreview);
    }

    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        // Check which request we're responding to
        if (requestCode == Tango.TANGO_INTENT_ACTIVITYCODE) {
            // Make sure the request was successful
            if (resultCode == RESULT_CANCELED) {
                Toast.makeText(this, "Motion Tracking Permissions Required!",
                        Toast.LENGTH_SHORT).show();
                finish();
            } else {
                startCameraPreview();
                mIsPermissionGranted = true;
            }
        }
    }

    // Camera Preview
    private void startCameraPreview() {
        // Connect to color camera
        tangoCameraPreview.connectToTangoCamera(mTango,
                TangoCameraIntrinsics.TANGO_CAMERA_RGBIR);
        // Use default configuration for Tango Service.
        TangoConfig config = mTango.getConfig(TangoConfig.CONFIG_TYPE_DEFAULT);
        config.putBoolean(TangoConfig.KEY_BOOLEAN_DEPTH, true);
        mTango.connect(config);
        mIsConnected = true;

        // No need to add any coordinate frame pairs since we are not using 
        // pose data. So just initialize.
        ArrayList<TangoCoordinateFramePair> framePairs = new ArrayList<TangoCoordinateFramePair>();
        mTango.connectListener(framePairs, new OnTangoUpdateListener() {
            @Override
            public void onPoseAvailable(TangoPoseData pose) {
                // We are not using OnPoseAvailable for this app
            }

            @Override
            public void onFrameAvailable(int cameraId) {

                // Check if the frame available is for the camera we want and
                // update its frame on the camera preview.
                if (cameraId == TangoCameraIntrinsics.TANGO_CAMERA_RGBIR) {
                    tangoCameraPreview.onFrameAvailable();
                }
            }

            @Override
            public void onXyzIjAvailable(TangoXyzIjData xyzIj) {
                // We are not using OnPoseAvailable for this app
            }

            @Override
            public void onTangoEvent(TangoEvent event) {
                // We are not using OnPoseAvailable for this app
            }
        });
    }

    @Override
    protected void onPause() {
        super.onPause();
        if(mIsConnected) {
            mTango.disconnect();
            tangoCameraPreview.disconnectFromTangoCamera();
            mIsConnected = false;
        }
    }

    @Override
    protected void onResume() {
        super.onResume();
        if (!mIsConnected && mIsPermissionGranted) {
            startCameraPreview();
        }
    }
}

解决方案

You can't use

tangoCameraPreview.connectToTangoCamera(mTango,TangoCameraIntrinsics.TANGO_CAMERA_RGBIR);

The Java API does not provide a connectToTangoCamera with the RGBIR, only Color and Fisheye camera (see here)

To display the a depth image instead of the color image, you have to compute the depth image by hand. Therefore you roughly have to:

1. Transform the point cloud with the pose at its given timestamp to the camera frame.
2. Project the point cloud points (x,y,z) to receive the pixel (x,y)
3. You need an array with the resolution of the camera frame (1240x720).
4. Fill up the array with 0 (black).
5. Store the z-value at its given pixel position (upsample your pixels, because the depth camera has only a resolution of 320x180)
6. In OpenGL you can than easly use that array for your texture.

For more details, I recommend you to look into the C example rgb-depth-sync

这篇关于Tango相机预览RGBIR的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！