人脸检测使用 Android Camera2 API 绘制圆圈 [英] Face detection & draw circle using Android Camera2 API

查看:48
本文介绍了人脸检测使用 Android Camera2 API 绘制圆圈的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

目前我正在尝试将 Camera2.Face 转换为实际视图的矩形,以便在 Camera2 API 检测到的面部上绘制圆圈.

我可以通过以下代码将人脸数量及其数据获取到回调中:

private CameraCaptureSession.CaptureCallback mCaptureCallback= 新的 CameraCaptureSession.CaptureCallback() {私有无效过程(CaptureResult 结果){整数模式 = result.get(CaptureResult.STATISTICS_FACE_DETECT_MODE);Face [] faces = result.get(CaptureResult.STATISTICS_FACES);if(faces != null && mode != null)Log.e("tag", "faces:" + faces.length + ", mode :" + mode );}@覆盖public void onCaptureProgressed(CameraCaptureSession session, CaptureRequest request, CaptureResult partialResult) {过程(部分结果);}@覆盖public void onCaptureCompleted(CameraCaptureSession 会话,CaptureRequest 请求,TotalCaptureResult 结果){过程(结果);}}

到目前为止,我尝试使用以下代码将 Face rect 转换为实际视图坐标(似乎不起作用):

/*** 来自 CameraCaptureSession.CaptureCallback 的回调*/@覆盖public void onFaceDetection(Face[] faces) {如果(mCameraView != null){setFaceDetectionMatrix();setFaceDetectionLayout(faces);}}/*** 此方法获取矩阵中人脸的缩放值*/私有无效 setFaceDetectionMatrix() {//人脸检测矩阵mFaceDetectionMatrix = 新矩阵();//前置摄像头需要镜子.boolean mirror = mCameraView.getFacing() == CameraView.FACING_FRONT;mFaceDetectionMatrix.setScale(镜像?-1:1、1);mFaceDetectionMatrix.postRotate(mCameraDisplayOrientation);矩形 activeArraySizeRect = mCameraView.getCameraCharacteristics().get(CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE);Log.i("Test", "activeArraySizeRect1: (" + activeArraySizeRect + ") -> " + activeArraySizeRect.width() + ", " + activeArraySizeRect.height());Log.i("Test", "activeArraySizeRect2:" + cameraOverlayDrawingView.getWidth() + ", " + cameraOverlayDrawingView.getHeight());浮动 s1 = cameraOverlayDrawingView.getWidth()/activeArraySizeRect.width();float s2 = cameraOverlayDrawingView.getHeight()/activeArraySizeRect.height();mFaceDetectionMatrix.postScale(s1, s2);mFaceDetectionMatrix.postTranslate(cameraOverlayDrawingView.getWidth()/2, cameraOverlayDrawingView.getHeight()/2);}/*** 此方法设置用于翻译 rect 的矩阵*/私有无效 setFaceDetectionLayout(Face[] faces) {if (faces.length == 0) {cameraOverlayDrawingView.setHaveFaces(false, null);} else if (faces.length > 0) {列表<矩形>faceRects;faceRects = new ArrayList<>();for (int i = 0; i  50) {int left = faces[i].getBounds().left;int top = faces[i].getBounds().top;int right = faces[i].getBounds().right;int bottom = faces[i].getBounds().bottom;Rect uRect = new Rect(left, top, right, bottom);RectF rectF = 新的 RectF(uRect);mFaceDetectionMatrix.mapRect(rectF);uRect.set((int) rectF.left, (int) rectF.top, (int) rectF.right, (int) rectF.bottom);Log.i("Test", "Activity rect" + i + " bounds: " + uRect);faceRects.add(uRect);}}cameraOverlayDrawingView.setHaveFaces(true, faceRects);}}

解决方案

NEW:我已经管理了我所有的手机轮换.我猜 offsetDxDy 取决于我的布局,但如果我必须告诉你真相,我不知道为什么我把值设置为 100.它在我的华为 P9 上运行良好,我以经验的方式找到了它.我仍然没有尝试确定是依赖于我的手机还是依赖于我的 XML 布局,或者两者都依赖.

无论如何,现在已经找到矩阵,因此您可以调整它们,使它们能够满足您的需求.

注意:我的 setRotation 不是那么通用,因为我没有对其进行参数化

intorientationOffset = mCameraCharacteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);

您可以尝试这样做,以便使用 SENSOR_ORIENTATION 的完整通用代码与本示例中的 270 不同.

因此,此代码适用于具有方向为 270 的硬件相机传感器的手机.

华为 P9 拥有它.

只是为了让您了解如何将旋转绑定到硬件传感器方向,这也适用于我的 P9(但我没有任何其他硬件可以测试)

if (mSwappedDimensions) {//显示旋转 0mFaceDetectionMatrix.setRotate(orientationOffset);mFaceDetectionMatrix.postScale(镜像?-s1:s1,s2);mFaceDetectionMatrix.postTranslate(mPreviewSize.getHeight() + offsetDxDy, mPreviewSize.getWidth() + offsetDxDy);} 别的 {//显示旋转 90 e 270if (displayRotation == Surface.ROTATION_90) {mFaceDetectionMatrix.setRotate(orientationOffset + 90);mFaceDetectionMatrix.postScale(镜像?-s1:s1,s2);mFaceDetectionMatrix.postTranslate(mPreviewSize.getWidth() + offsetDxDy, -offsetDxDy);} else if (displayRotation == Surface.ROTATION_270) {mFaceDetectionMatrix.setRotate(orientationOffset + 270);mFaceDetectionMatrix.postScale(镜像?-s1:s1,s2);mFaceDetectionMatrix.postTranslate(-offsetDxDy, mPreviewSize.getHeight() + offsetDxDy);}}

这是我的最终代码(也可在 GitHub 上找到)

intorientationOffset = mCameraCharacteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);矩形 activeArraySizeRect = mCameraCharacteristics.get(CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE);//人脸检测矩阵mFaceDetectionMatrix = 新矩阵();Log.i("Test", "activeArraySizeRect1: (" + activeArraySizeRect + ") -> " + activeArraySizeRect.width() + ", " + activeArraySizeRect.height());Log.i("Test", "activeArraySizeRect2:" + mPreviewSize.getWidth() + ", " + mPreviewSize.getHeight());浮动 s1 = mPreviewSize.getWidth()/(float)activeArraySizeRect.width();float s2 = mPreviewSize.getHeight()/(float)activeArraySizeRect.height();//float s1 = mOverlayView.getWidth();//float s2 = mOverlayView.getHeight();boolean mirror = (面对 == CameraCharacteristics.LENS_FACING_FRONT);//我们总是使用前置摄像头boolean weAreinPortrait = true;int offsetDxDy = 100;如果(mSwappedDimensions){//显示旋转 0mFaceDetectionMatrix.setRotate(270);mFaceDetectionMatrix.postScale(镜像?-s1:s1,s2);mFaceDetectionMatrix.postTranslate(mPreviewSize.getHeight() + offsetDxDy, mPreviewSize.getWidth() + offsetDxDy);} 别的 {//显示旋转 90 e 270if (displayRotation == Surface.ROTATION_90) {mFaceDetectionMatrix.setRotate(0);mFaceDetectionMatrix.postScale(镜像?-s1:s1,s2);mFaceDetectionMatrix.postTranslate(mPreviewSize.getWidth() + offsetDxDy, -offsetDxDy);} else if (displayRotation == Surface.ROTATION_270) {mFaceDetectionMatrix.setRotate(180);mFaceDetectionMatrix.postScale(镜像?-s1:s1,s2);mFaceDetectionMatrix.postTranslate(-offsetDxDy, mPreviewSize.getHeight() + offsetDxDy);}}

<块引用>

这是您所在的公共 github 存储库可以找到代码:

无论如何只是为了给你一些理论,你正在做的是一个 2D 平面变换.我的意思是你有一个平面(硬件传感器),你必须在预览平面上重新映射该平面上的对象.

所以你必须照顾:

  • 旋转:这取决于您的硬件传感器旋转和手机旋转.
  • 镜像:水平镜像取决于您是否使用前置摄像头以及垂直镜像取决于手机旋转).镜像是通过缩放矩阵中的-"符号完成的.
  • 平移:这取决于您的对象通过旋转(这也取决于您正在处理的旋转中心)和平移放置的位置.所以你必须在你的预览视图中替换你的对象.

数学理论

前段时间我也在我的博客中写了一些技术文章,但它们是意大利语.

Currently I am trying to convert Camera2.Face to actual view's rect in order to draw circle over the face detected by the Camera2 API.

I am able to get number of faces and its data into Callback by below code:

private CameraCaptureSession.CaptureCallback mCaptureCallback
= new CameraCaptureSession.CaptureCallback() {
    private void process(CaptureResult result) {
        Integer mode = result.get(CaptureResult.STATISTICS_FACE_DETECT_MODE);
        Face [] faces = result.get(CaptureResult.STATISTICS_FACES);
        if(faces != null && mode != null)
            Log.e("tag", "faces : " + faces.length + " , mode : " + mode ); 
    }

    @Override
    public void onCaptureProgressed(CameraCaptureSession session, CaptureRequest request, CaptureResult partialResult) {
        process(partialResult);
    }

    @Override
    public void onCaptureCompleted(CameraCaptureSession session, CaptureRequest request, TotalCaptureResult result) {
        process(result);
    }
}

I tried below code so far to convert Face rect to actual view co-ordinates(seems like it is not working):

/**
* Callback from the CameraCaptureSession.CaptureCallback
*/
@Override
public void onFaceDetection(Face[] faces) {
    if (mCameraView != null) {
        setFaceDetectionMatrix();
        setFaceDetectionLayout(faces);
    }
}

/**
 * This method gets the scaling values of the face in matrix
 */
private void setFaceDetectionMatrix() {
    // Face Detection Matrix
    mFaceDetectionMatrix = new Matrix();
    // Need mirror for front camera.
    boolean mirror = mCameraView.getFacing() == CameraView.FACING_FRONT;
    mFaceDetectionMatrix.setScale(mirror ? -1 : 1, 1);
    mFaceDetectionMatrix.postRotate(mCameraDisplayOrientation);

    Rect activeArraySizeRect = mCameraView.getCameraCharacteristics().get(CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE);
    Log.i("Test", "activeArraySizeRect1: (" + activeArraySizeRect + ") -> " + activeArraySizeRect.width() + ", " + activeArraySizeRect.height());
    Log.i("Test", "activeArraySizeRect2: " + cameraOverlayDrawingView.getWidth() + ", " + cameraOverlayDrawingView.getHeight());
    float s1 = cameraOverlayDrawingView.getWidth() / activeArraySizeRect.width();
    float s2 = cameraOverlayDrawingView.getHeight() / activeArraySizeRect.height();
    mFaceDetectionMatrix.postScale(s1, s2);
    mFaceDetectionMatrix.postTranslate(cameraOverlayDrawingView.getWidth() / 2, cameraOverlayDrawingView.getHeight() / 2);
}

/**
 * This method set the matrix for translating rect
 */
private void setFaceDetectionLayout(Face[] faces) {
    if (faces.length == 0) {
        cameraOverlayDrawingView.setHaveFaces(false, null);
    } else if (faces.length > 0) {
        List<Rect> faceRects;
        faceRects = new ArrayList<>();
        for (int i = 0; i < faces.length; i++) {
            Log.i("Test", "Activity face" + i + " bounds: " + faces[i].getBounds());
            if (faces[i].getScore() > 50) {
                int left = faces[i].getBounds().left;
                int top = faces[i].getBounds().top;
                int right = faces[i].getBounds().right;
                int bottom = faces[i].getBounds().bottom;

                Rect uRect = new Rect(left, top, right, bottom);
                RectF rectF = new RectF(uRect);
                mFaceDetectionMatrix.mapRect(rectF);
                uRect.set((int) rectF.left, (int) rectF.top, (int) rectF.right, (int) rectF.bottom);
                Log.i("Test", "Activity rect" + i + " bounds: " + uRect);
                    faceRects.add(uRect);
            }
        }
        cameraOverlayDrawingView.setHaveFaces(true, faceRects);
    }
}

解决方案

NEW: I've manage all my phone rotations. The offsetDxDy I guess depends on my layout, but if I've to tell you the truth I don't know why I put a value of 100. It works well on my Huawei P9 and I've found it in an empirical way. I still not have tried to find out if depends on my phone or on my XML layout or both.

Anyway the Matrices now are found, so you could adapt them so that they can fit your needs.

Note: my setRotation is not so general, because I didn't parametrized it upon

int orientationOffset = mCameraCharacteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);

You can try do do it so that to have a full general code working with SENSOR_ORIENTATION different from the one of this example that is 270.

So this code works with a phone with an hardware camera sensor with orientation of 270.

The Huawei P9 has it.

Just to give you an idea of making the rotation bind to se HW sensor orientation that also works well on my P9 (but I don't have any other hardware to test)

if (mSwappedDimensions) {
    // Display Rotation 0
    mFaceDetectionMatrix.setRotate(orientationOffset);
    mFaceDetectionMatrix.postScale(mirror ? -s1 : s1, s2);
    mFaceDetectionMatrix.postTranslate(mPreviewSize.getHeight() + offsetDxDy, mPreviewSize.getWidth() + offsetDxDy);
} else {
    // Display Rotation 90 e 270
    if (displayRotation == Surface.ROTATION_90) {
        mFaceDetectionMatrix.setRotate(orientationOffset + 90);
        mFaceDetectionMatrix.postScale(mirror ? -s1 : s1, s2);
        mFaceDetectionMatrix.postTranslate(mPreviewSize.getWidth() + offsetDxDy, -offsetDxDy);
    } else if (displayRotation == Surface.ROTATION_270) {
        mFaceDetectionMatrix.setRotate(orientationOffset + 270);
        mFaceDetectionMatrix.postScale(mirror ? -s1 : s1, s2);
        mFaceDetectionMatrix.postTranslate(-offsetDxDy, mPreviewSize.getHeight() + offsetDxDy);
    }
}

Here my final code (also available on GitHub)

int orientationOffset = mCameraCharacteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);
Rect activeArraySizeRect = mCameraCharacteristics.get(CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE);

// Face Detection Matrix
mFaceDetectionMatrix = new Matrix();

Log.i("Test", "activeArraySizeRect1: (" + activeArraySizeRect + ") -> " + activeArraySizeRect.width() + ", " + activeArraySizeRect.height());
Log.i("Test", "activeArraySizeRect2: " + mPreviewSize.getWidth() + ", " + mPreviewSize.getHeight());
float s1 = mPreviewSize.getWidth() / (float)activeArraySizeRect.width();
float s2 = mPreviewSize.getHeight() / (float)activeArraySizeRect.height();
//float s1 = mOverlayView.getWidth();
//float s2 = mOverlayView.getHeight();
boolean mirror = (facing == CameraCharacteristics.LENS_FACING_FRONT); // we always use front face camera
boolean weAreinPortrait = true;
int offsetDxDy = 100;
if (mSwappedDimensions) {
    // Display Rotation 0
    mFaceDetectionMatrix.setRotate(270);
    mFaceDetectionMatrix.postScale(mirror ? -s1 : s1, s2);
    mFaceDetectionMatrix.postTranslate(mPreviewSize.getHeight() + offsetDxDy, mPreviewSize.getWidth() + offsetDxDy);
} else {
    // Display Rotation 90 e 270
    if (displayRotation == Surface.ROTATION_90) {
        mFaceDetectionMatrix.setRotate(0);
        mFaceDetectionMatrix.postScale(mirror ? -s1 : s1, s2);
        mFaceDetectionMatrix.postTranslate(mPreviewSize.getWidth() + offsetDxDy, -offsetDxDy);
    } else if (displayRotation == Surface.ROTATION_270) {
        mFaceDetectionMatrix.setRotate(180);
        mFaceDetectionMatrix.postScale(mirror ? -s1 : s1, s2);
        mFaceDetectionMatrix.postTranslate(-offsetDxDy, mPreviewSize.getHeight() + offsetDxDy);
    }
}

This is the public github repo where you can find the code: https://github.com/shadowsheep1/android-camera2-api-face-recon. Hope it could help you.

Anyway just to give you also some theory, what you are doing is a 2D plane transformation. I mean you have a plane (the HW Sensor) and you have to remap the object on that plane on your preview plane.

So you have to take care of:

  • Rotation: That depends on your HW Sensor rotation and the Phone Rotation.
  • Mirroring: Horizontal mirroring that depends if you are using the front face camera or not and the Vertical mirroring that depends on the phone rotation). Mirroring is done with a '-' sign in the scaling matrix.
  • Translation: That depends where your object has been placed by the rotation (that depends also from which rotation center your are dealing with) and translation. So you have to replace in your preview View your objects.

Math Theory

I've also write some technical post in my blog some time ago but they are in Italian.

这篇关于人脸检测使用 Android Camera2 API 绘制圆圈的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆