opencv4android模板匹配使用相机 [英] opencv4android template matching using camera

问题描述

我已下载并成功运行进行仿射不变SIFT匹配或正常SIFT（存在OpenCV实现）。然而，
，因为这些是在C ++中，你可能想使用JNI从Android设备上的Java调用它。这可能是从4个符号中检测卡片套装的最佳方式。

检测和识别卡片上的数字/字母的另一个选项是使用文本检测器，例如< a href =http://docs.opencv.org/3.0-beta/modules/text/doc/erfilter.html =nofollow> MSER ，然后在感兴趣的区域上显示文本识别器通过MSER滤波器。

在任何情况下，您都不可能从图片中显示的图片类型中产生最好的结果。使用第一种方法，您可以获得满意的正面图像的可接受的性能。

I have downloaded and successfully run the example provided in opencv4android sdk.

I am able to simply display the camera frames without any processing,

 public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
     return inputFrame.rgba();
 }

I want to process live frame with some predefined image template to recognize that template. I have taken reference from this post and implemented accordingly. But I get black screen only.

private Mat mCameraMat = new Mat();
private Mat mTemplateMat;

 public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
    mCameraMat = inputFrame.rgba();
    initialize();

     int match_method = Imgproc.TM_SQDIFF;

        // Create the result matrix
        int result_cols = mCameraMat.cols() - mTemplateMat.cols() + 1;
        int result_rows = mCameraMat.rows() - mTemplateMat.rows() + 1;
        Log.d(TAG, " mCameraMat cols "+mCameraMat.cols());
        Log.d(TAG, " mCameraMat rows "+mCameraMat.rows());
        Log.d(TAG, " mTemplateMat cols "+mTemplateMat.cols());
        Log.d(TAG, " mTemplateMat rows "+mTemplateMat.rows());

       Mat result = new Mat(result_rows, result_cols, CvType.CV_32F);

        // Do the Matching and Normalize
        Imgproc.matchTemplate(mCameraMat, mTemplateMat, result, match_method);



        Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat());

        // Localizing the best match with minMaxLoc
        MinMaxLocResult mmr = Core.minMaxLoc(result);

        Point matchLoc;
        if (match_method == Imgproc.TM_SQDIFF || match_method == Imgproc.TM_SQDIFF_NORMED) {
            matchLoc = mmr.minLoc;
        } else {
            matchLoc = mmr.maxLoc;
        }

        Rect roi = new Rect((int) matchLoc.x, (int) matchLoc.y, mTemplateMat.cols(), mTemplateMat.rows());
        Core.rectangle(mCameraMat, new Point(roi.x, roi.y), new Point(roi.width - 2, roi.height - 2), new Scalar(255, 0, 0, 255), 2);           
 return result;
}

public void initialize(){

    try {
        if (mCameraMat.empty())
            return;
        if(mTemplateMat == null){
            Mat temp = Utils.loadResource(Tutorial1Activity.this, R.drawable.icon);
            mTemplateMat = new Mat(temp.size(), CvType.CV_32F);
            Imgproc.cvtColor(temp, mTemplateMat, Imgproc.COLOR_BGR2RGBA);
            Log.d(TAG, "initialize mTemplateMat cols "+mTemplateMat.cols());
            Log.d(TAG, "initialize mTemplateMat rows "+mTemplateMat.rows());
        }
    } catch (IOException e) {
        e.printStackTrace();
    }
}

Note:

My ultimate goal is to recognize the playing cards from live camera. Kindly suggest best approach. Should I use image templates or any other thing to make things faster?

This is how I want to recognize multiple cards from live camera:

Result should be: ♠A ♠K ♠Q ♠J ♠10 when camera preview seems like below

解决方案

Template matching is unlikely to be the best approach here.

1. Try aSIFT to do an affine invariant SIFT matching or a normal SIFT (OpenCV implementation exists). However, since these are in C++, you may want to use JNI to make calls to it from Java on an Android device. This is probably the best way to detect the suit of the card from the 4 symbols.
2. Another option to detect and recognize the numbers/alphabets on the cards is to use a text detector like MSER and then a text recognizer on the regions of interest indicated by the MSER filter.

In any case, you are unlikely to be able produce the best of results from the kind of image you've shown in the image. You may be able to get acceptable performance for full frontal, upright images with the first method.

这篇关于opencv4android模板匹配使用相机的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！