使用相机进行opencv4android模板匹配 [英] opencv4android template matching using camera

问题描述

我已下载并成功运行 进行仿射不变 SIFT 匹配或普通的 SIFT(存在 OpenCV 实现).然而，由于这些是 C++ 语言，您可能希望使用 JNI 在 Android 设备上从 Java 调用它.这可能是从 4 个符号中检测卡片花色的最佳方法.

检测和识别卡片上的数字/字母的另一种方法是使用文本检测器，例如 MSER 然后是 MSER 过滤器指示的感兴趣区域上的文本识别器.

在任何情况下，您都不太可能从您在图片中显示的那种图片中获得最佳效果.使用第一种方法，您可能能够获得可接受的完整正面、直立图像的性能.

I have downloaded and successfully run the example provided in opencv4android sdk.

I am able to simply display the camera frames without any processing,

 public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
     return inputFrame.rgba();
 }

I want to process live frame with some predefined image template to recognize that template. I have taken reference from this post and implemented accordingly. But I get black screen only.

private Mat mCameraMat = new Mat();
private Mat mTemplateMat;

 public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
    mCameraMat = inputFrame.rgba();
    initialize();

     int match_method = Imgproc.TM_SQDIFF;

        // Create the result matrix
        int result_cols = mCameraMat.cols() - mTemplateMat.cols() + 1;
        int result_rows = mCameraMat.rows() - mTemplateMat.rows() + 1;
        Log.d(TAG, " mCameraMat cols "+mCameraMat.cols());
        Log.d(TAG, " mCameraMat rows "+mCameraMat.rows());
        Log.d(TAG, " mTemplateMat cols "+mTemplateMat.cols());
        Log.d(TAG, " mTemplateMat rows "+mTemplateMat.rows());

       Mat result = new Mat(result_rows, result_cols, CvType.CV_32F);

        // Do the Matching and Normalize
        Imgproc.matchTemplate(mCameraMat, mTemplateMat, result, match_method);



        Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat());

        // Localizing the best match with minMaxLoc
        MinMaxLocResult mmr = Core.minMaxLoc(result);

        Point matchLoc;
        if (match_method == Imgproc.TM_SQDIFF || match_method == Imgproc.TM_SQDIFF_NORMED) {
            matchLoc = mmr.minLoc;
        } else {
            matchLoc = mmr.maxLoc;
        }

        Rect roi = new Rect((int) matchLoc.x, (int) matchLoc.y, mTemplateMat.cols(), mTemplateMat.rows());
        Core.rectangle(mCameraMat, new Point(roi.x, roi.y), new Point(roi.width - 2, roi.height - 2), new Scalar(255, 0, 0, 255), 2);           
 return result;
}

public void initialize(){

    try {
        if (mCameraMat.empty())
            return;
        if(mTemplateMat == null){
            Mat temp = Utils.loadResource(Tutorial1Activity.this, R.drawable.icon);
            mTemplateMat = new Mat(temp.size(), CvType.CV_32F);
            Imgproc.cvtColor(temp, mTemplateMat, Imgproc.COLOR_BGR2RGBA);
            Log.d(TAG, "initialize mTemplateMat cols "+mTemplateMat.cols());
            Log.d(TAG, "initialize mTemplateMat rows "+mTemplateMat.rows());
        }
    } catch (IOException e) {
        e.printStackTrace();
    }
}

Note:

My ultimate goal is to recognize the playing cards from live camera. Kindly suggest best approach. Should I use image templates or any other thing to make things faster?

This is how I want to recognize multiple cards from live camera:

Result should be: ♠A ♠K ♠Q ♠J ♠10 when camera preview seems like below

解决方案

Template matching is unlikely to be the best approach here.

1. Try aSIFT to do an affine invariant SIFT matching or a normal SIFT (OpenCV implementation exists). However, since these are in C++, you may want to use JNI to make calls to it from Java on an Android device. This is probably the best way to detect the suit of the card from the 4 symbols.
2. Another option to detect and recognize the numbers/alphabets on the cards is to use a text detector like MSER and then a text recognizer on the regions of interest indicated by the MSER filter.

In any case, you are unlikely to be able produce the best of results from the kind of image you've shown in the image. You may be able to get acceptable performance for full frontal, upright images with the first method.

这篇关于使用相机进行opencv4android模板匹配的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！