OpenCV Java:从图像中提取卡 [英] OpenCV Java : Card Extraction from Image

问题描述

我正在尝试使用OpenCV和Java进行一些图像处理，以从图像中提取卡.

I am trying to implement some image processing using OpenCV and Java to extract a card out of an image.

以下是我的方法:

1. 转换为BGR图片
2. 转换为灰色图像
3. 应用高斯模糊
4. 应用Canny Edge检测
5. 扩张
6. 找到轮廓
7. 找到最大的轮廓
8. 使用roxPolyDP查找最大轮廓的角
9. 沿着最大轮廓获取裁剪图像的自顶向下视图

在第8步中，我遇到了一些问题，因为我没有获得适当的角/顶点.以下示例图像显示了该场景:

At step no 8, I am facing some issues, as I am not getting the appropriate corners/vertices. Following sample images shows the scenario :

原始图像

在边缘检测和扩张之后. (要获得合适的边缘该怎么做?这里是折边.无法使Hough变换正常工作)

After edge detection and dilation. (What is to be done to get appropriate edges?? Here I've got broken edges. Could not get Hough transform working)

找到顶点后. (以绿色显示)

After finding vertices. (shown in green)

以下是代码:

System.loadLibrary( Core.NATIVE_LIBRARY_NAME );

         //load Image
         File input = new File("card4.png");
         BufferedImage image = ImageIO.read(input); 
         byte[] data = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();

         //put read image to Mat
         mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC3); //original Mat
         mat.put(0, 0, data);
         mat_f = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC3); //for storing manipulated Mat

         //conversion to grayscale, blurring and edge detection
         Imgproc.cvtColor(mat, mat_f, Imgproc.COLOR_RGB2BGR);
         Imgproc.cvtColor(mat_f, mat_f, Imgproc.COLOR_RGB2GRAY);
         Imgproc.GaussianBlur(mat_f, mat_f, new Size(13,13), 0);             
         Imgproc.Canny(mat_f, mat_f, 300, 600, 5, true);
         Imgproc.dilate(mat_f, mat_f, new Mat(), new Point(-1, -1), 2);
         Imgcodecs.imwrite("D:\\JAVA\\Image_Proc\\CVTest1.jpg",mat_f);

         //finding contours
         List<MatOfPoint> contours = new ArrayList<MatOfPoint>();    
         Mat hierarchy = new Mat();
         Imgproc.findContours(mat_f, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
         double maxArea=0;
         int maxAreaIdx=0;

         //finding largest contour
         for (int idx = 0; idx != contours.size(); ++idx)
         {
               Mat contour = contours.get(idx);
               double contourarea = Imgproc.contourArea(contour);
               if (contourarea > maxArea)
               {
                   maxArea = contourarea;
                   maxAreaIdx = idx;
               }

          }

            //Rect rect = Imgproc.boundingRect(contours.get(maxAreaIdx));
            //Imgproc.rectangle(mat, new Point(rect.x,rect.y), new Point(rect.x+rect.width,rect.y+rect.height),new Scalar(0,0,255),7);
           // mat = mat.submat(rect.y, rect.y + rect.height, rect.x, rect.x + rect.width);


          //Polygon approximation
          MatOfPoint2f approxCurve = new MatOfPoint2f();
          MatOfPoint2f oriCurve = new MatOfPoint2f(contours.get(maxAreaIdx).toArray());
          Imgproc.approxPolyDP(oriCurve, approxCurve, 6.0, true);

          //drawing red markers at vertices
          Point [] array = approxCurve.toArray();
          for(int i=0; i < array.length;i++) {
             Imgproc.circle(mat, array[i], 2, new Scalar(0, 255, 0), 5);
          }
          Imgcodecs.imwrite("D:\\JAVA\\Image_Proc\\CVTest.jpg",mat);

寻求帮助以获取适当的角顶点... 预先感谢..

Seeking help in getting the appropriate corner vertices... Thanks in advance..

推荐答案

为了使用您的方法来存档良好的结果，您的卡必须包含4个角.但是我更喜欢使用HoughLine方法来完成此任务.

第1步:调整图像大小以提高性能

In order to archive the good result using your approach then your cards have to contain 4 corners. But i prefer to use the HoughLine approach for this task.

Step 1: Resize image for higher performance

• 将图像转换为灰度
• 对图像进行模糊处理以清除噪波
• 使用Canny滤镜的边缘检测

您可以使用膨胀使白色变大，用于下一步

You can use the dilation for make the white bigger for the next step

• 查找图像轮廓
• 从轮廓列表中获取最大轮廓
• 获取它的凸包
• 使用roxPolyDP简化凸包(这应为四边形)
• 从现在开始，您可以绘制轮廓以在还原比例后获得矩形
• 从四边形可以得到4个角.
• 寻找单应性
• 使用计算的单应矩阵对输入图像进行变形

这是Java中的示例代码

Here is sample code in Java

    // STEP 1: Resize input image to img_proc to reduce computation
    double ratio = DOWNSCALE_IMAGE_SIZE / Math.max(frame.width(), frame.height());
    Size downscaledSize = new Size(frame.width() * ratio, frame.height() * ratio);
    Mat dst = new Mat(downscaledSize, frame.type());
    Imgproc.resize(frame, dst, downscaledSize);
    Mat grayImage = new Mat();
    Mat detectedEdges = new Mat();
    // STEP 2: convert to grayscale
    Imgproc.cvtColor(dst, grayImage, Imgproc.COLOR_BGR2GRAY);
    // STEP 3: try to filter text inside document
    Imgproc.medianBlur(grayImage, detectedEdges, 9);
    // STEP 4: Edge detection
    Mat edges = new Mat();
    // Imgproc.erode(edges, edges, new Mat());
    // Imgproc.dilate(edges, edges, new Mat(), new Point(-1, -1), 1); // 1
    // canny detector, with ratio of lower:upper threshold of 3:1
    Imgproc.Canny(detectedEdges, edges, this.threshold.getValue(), this.threshold.getValue() * 3, 3, true);
    // STEP 5: makes the object in white bigger to join nearby lines
    Imgproc.dilate(edges, edges, new Mat(), new Point(-1, -1), 1); // 1
    Image imageToShow = Utils.mat2Image(edges);
    updateImageView(cannyFrame, imageToShow);
    // STEP 6: Compute the contours
    List<MatOfPoint> contours = new ArrayList<>();
    Imgproc.findContours(edges, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
    // STEP 7: Sort the contours by length and only keep the largest one
    MatOfPoint largestContour = getMaxContour(contours);
    // STEP 8: Generate the convex hull of this contour
    Mat convexHullMask = Mat.zeros(frame.rows(), frame.cols(), frame.type());
    MatOfInt hullInt = new MatOfInt();
    Imgproc.convexHull(largestContour, hullInt);
    MatOfPoint hullPoint = OpenCVUtil.getNewContourFromIndices(largestContour, hullInt);
    // STEP 9: Use approxPolyDP to simplify the convex hull (this should give a quadrilateral)
    MatOfPoint2f polygon = new MatOfPoint2f();
    Imgproc.approxPolyDP(OpenCVUtil.convert(hullPoint), polygon, 20, true);
    List<MatOfPoint> tmp = new ArrayList<>();
    tmp.add(OpenCVUtil.convert(polygon));
    restoreScaleMatOfPoint(tmp, ratio);
    Imgproc.drawContours(convexHullMask, tmp, 0, new Scalar(25, 25, 255), 2);
    // Image extractImageToShow = Utils.mat2Image(convexHullMask);
    // updateImageView(extractFrame, extractImageToShow);
    MatOfPoint2f finalCorners = new MatOfPoint2f();
    Point[] tmpPoints = polygon.toArray();
    for (Point point : tmpPoints) {
        point.x = point.x / ratio;
        point.y = point.y / ratio;
    }
    finalCorners.fromArray(tmpPoints);
    boolean clockwise = true;
    double currentThreshold = this.threshold.getValue();
    if (finalCorners.toArray().length == 4) {
        Size size = getRectangleSize(finalCorners);
        Mat result = Mat.zeros(size, frame.type());
        // STEP 10: Homography: Use findHomography to find the affine transformation of your paper sheet
        Mat homography = new Mat();
        MatOfPoint2f dstPoints = new MatOfPoint2f();
        Point[] arrDstPoints = { new Point(result.cols(), result.rows()), new Point(0, result.rows()), new Point(0, 0), new Point(result.cols(), 0) };
        dstPoints.fromArray(arrDstPoints);
        homography = Calib3d.findHomography(finalCorners, dstPoints);

        // STEP 11: Warp the input image using the computed homography matrix
        Imgproc.warpPerspective(frame, result, homography, size);
    }

这篇关于OpenCV Java:从图像中提取卡的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！