OpenCV warpPerspective-如何知道目标图像大小? [英] OpenCV warpPerspective - how to know destination image size?

问题描述

好吧，我必须承认我是OpenCV的新手，也是我的MATLAB/lin.代数知识可能会引入偏见.但是我想做的事情很简单，尽管我仍然没有找到答案.

Ok, I have to admit that I am a novice to OpenCV and that my MATLAB/lin. Algebra knowledge might be introducing a bias. But what I want to do is really simple, while I still did not manage to find an answer.

当尝试在透视变换下校正图像(或图像的一部分)时，基本上执行两个步骤(假设您具有定义变形对象的4个点):

When trying to rectify an image (or part of an image) under a perspective transformation, you basically perform two steps (assuming you have the 4 points that define the distorted object):

1. 找到一些完美的矩形和变形的形状之间的转换(在OpenCV中，通过findHomography()或getPerspectiveTransform()-为什么这两个在相同点上的不同操作又是一个故事，同样令人沮丧)；这给了我们矩阵T.
2. 将T的倒数应用于最初变形的形状，以将其转换为矩形(在OpenCV中，这是通过warpPerspective()完成的.)

1. find the transformation between some perfect rectangle and the distorted shape (in OpenCV, via findHomography() or getPerspectiveTransform() - why those two operate differently on the same points is another story, also frustrating); this gives us a matrix T.
2. apply the inverse of T to the initially distorted shape to transform it into a rectangle (in OpenCV, this is done with warpPerspective()).

现在，最后一个功能(warpPerspective())要求用户指定目标图像的大小.

Now, this last function (warpPerspective()) asks the user to specify the size of the destination image.

我的问题是用户应该如何事先知道该大小是多少.低级的实现方法是将变换T应用于找到对象的图像的角点，从而确保您不会脱离新变换的形状的范围. 但是，即使您从T中取出矩阵并将其手动应用于这些点，结果看起来也很奇怪.

My question is how the users should know beforehand what that size would be. The low-level way of doing it is simply applying the transformation T to the corner points of the image in which the object is found, thus guaranteeing that you don't get out of the bounds with the newly transformed shape. However, even if you take the matrix out of T and apply it manually to those points, the result looks weird.

在OpenCV中有没有办法做到这一点?谢谢！

Is there a way to do this in OpenCV? Thanks!

P.S.下面是一些代码:

P.S. Below is some code:

float leftX, lowerY, rightX, higherY;   

float minX = std::numeric_limits<float>::max(), maxX = std::numeric_limits<float>::min(), minY = std::numeric_limits<float>::max(), maxY = std::numeric_limits<float>::min();

Mat value, pt;
for(int i=0; i<4; i++)
{
    switch(i)
    {
        case 0:
            pt = (Mat_<float>(3, 1) << 1.00,1.00,1.00);                         
            break;
        case 1:
            pt = (Mat_<float>(3, 1) << srcIm.cols,1.00,1.00);
            break;
        case 2:
            pt = (Mat_<float>(3, 1) << 1.00,srcIm.rows,1.00);
            break;
        case 3:
            pt = (Mat_<float>(3, 1) << srcIm.cols,srcIm.rows,1.00);
            break;
        default:
            cerr << "Wrong switch." << endl;
            break;
    }               
    value = invH*pt;    
    value /= value.at<float>(2);        
    minX = min(minX,value.at<float>(0));
    maxX = max(maxX,value.at<float>(0));
    minY = min(minY,value.at<float>(1));
    maxY = max(maxY,value.at<float>(1));
}
leftX = std::min<float>(1.00,-minX);
lowerY = std::min<float>(1.00,-minY);
rightX = max(srcIm.cols-minX,maxX-minX);
higherY = max(srcIm.rows-minY,maxY-minY);

warpPerspective(srcIm, dstIm, H, Size(rightX-leftX,higherY-lowerY), cv::INTER_CUBIC);

更新:也许我的结果看起来不好，因为我使用的矩阵是错误的.由于我无法观察getPerspectiveTransform()内部发生的事情，因此我不知道该矩阵的计算方式，但是它具有一些非常小的值和非常大的值，这使我认为它们是垃圾. 这就是我从T获得数据的方式:

UPDATE: Perhaps my results do not look good because the matrix I'm using is wrong. As I cannot observe what's happening inside getPerspectiveTransform(), I cannot know how this matrix is computed, but it has some very small and very large values, which makes me think they are garbage. This is the way I obtain the data from T:

for(int row=0;row<3;row++)
    for(int col=0;col<3;col++)
        T.at<float>(row,col) = ((float*)(H.data + (size_t)H.step*row))[col];

(尽管getPerspectiveTransform()的输出矩阵为3x3，但尝试直接通过T.at<float>(row,col)访问其值会导致分段错误.)

(Although the output matrix from getPerspectiveTransform() is 3x3, trying to access its values directly via T.at<float>(row,col) leads to a segmentation fault.)

这是正确的方法吗?也许这就是为什么出现原始问题的原因，因为我没有获得正确的矩阵...

Is this the right way to do it? Perhaps this is why the original issue arises, because I do not get the correct matrix...

推荐答案

如果在调用warpPerspective之前知道图像的大小，则可以获取其四个角的坐标，并用PerspectiveTransform对其进行变换，以了解如何它们将在转换时显示出来.据推测，它们将不再形成漂亮的矩形，因此您可能需要计算最小值和最大值以获取边界框.然后，此边界框的大小就是您想要的目标大小. (此外，如果任何一个角都降到零以下，也不要忘记根据需要平移该框.)这是一个Python示例，该示例使用warpPerspective将变换后的图像叠加在其顶部.

If you know what the size of your image was before you call warpPerspective, then you can take the coordinates of its four corners and transform them with perspectiveTransform to see how they will turn out when they are transformed. Presumably, they will no longer form a nice rectangle, so you will probably want to compute the mins and maxs to obtain a bounding box. Then, the size of this bounding box is the destination size you want. (Also, don't forget to translate the box as needed if any of the corners dip below zero.) Here is a Python example that uses warpPerspective to blit a transformed image on top of itself.

from typing import Tuple
import cv2
import numpy as np
import math

# Input: a source image and perspective transform
# Output: a warped image and 2 translation terms
def perspective_warp(image: np.ndarray, transform: np.ndarray) -> Tuple[np.ndarray, int, int]:
    h, w = image.shape[:2]
    corners_bef = np.float32([[0, 0], [w, 0], [w, h], [0, h]]).reshape(-1, 1, 2)
    corners_aft = cv2.perspectiveTransform(corners_bef, transform)
    xmin = math.floor(corners_aft[:, 0, 0].min())
    ymin = math.floor(corners_aft[:, 0, 1].min())
    xmax = math.ceil(corners_aft[:, 0, 0].max())
    ymax = math.ceil(corners_aft[:, 0, 1].max())
    x_adj = math.floor(xmin - corners_aft[0, 0, 0])
    y_adj = math.floor(ymin - corners_aft[0, 0, 1])
    translate = np.eye(3)
    translate[0, 2] = -xmin
    translate[1, 2] = -ymin
    corrected_transform = np.matmul(translate, transform)
    return cv2.warpPerspective(image, corrected_transform, (math.ceil(xmax - xmin), math.ceil(ymax - ymin))), x_adj, y_adj

# Just like perspective_warp, but it also returns an alpha mask that can be used for blitting
def perspective_warp_with_mask(image: np.ndarray, transform: np.ndarray) -> Tuple[np.ndarray, np.ndarray, int, int]:
    mask_in = np.empty(image.shape, dtype = np.uint8)
    mask_in.fill(255)
    output, x_adj, y_adj = perspective_warp(image, transform)
    mask, _, _ = perspective_warp(mask_in, transform)
    return output, mask, x_adj, y_adj

# alpha_blits src onto dest according to the alpha values in mask at location (x, y),
# ignoring any parts that do not overlap
def alpha_blit(dest: np.ndarray, src: np.ndarray, mask: np.ndarray, x: int, y: int) -> None:
    dl = max(x, 0)
    dt = max(y, 0)
    sl = max(-x, 0)
    st = max(-y, 0)
    sr = max(sl, min(src.shape[1], dest.shape[1] - x))
    sb = max(st, min(src.shape[0], dest.shape[0] - y))
    dr = dl + sr - sl
    db = dt + sb - st
    m = mask[st:sb, sl:sr]
    dest[dt:db, dl:dr] = (dest[dt:db, dl:dr].astype(np.float) * (255 - m) + src[st:sb, sl:sr].astype(np.float) * m) / 255

# blits a perspective-warped src image onto dest
def perspective_blit(dest: np.ndarray, src: np.ndarray, transform: np.ndarray) -> None:
    blitme, mask, x_adj, y_adj = perspective_warp_with_mask(src, transform)
    cv2.imwrite("blitme.png", blitme)
    alpha_blit(dest, blitme, mask, int(transform[0, 2] + x_adj), int(transform[1, 2] + y_adj))


# Read an input image
image: np.array = cv2.imread('input.jpg')

# Make a perspective transform
h, w = image.shape[:2]
corners_in = np.float32([[[0, 0]], [[w, 0]], [[w, h]], [[0, h]]])
corners_out = np.float32([[[100, 100]], [[300, -100]], [[500, 300]], [[-50, 500]]])
transform = cv2.getPerspectiveTransform(corners_in, corners_out)

# Blit the warped image on top of the original
perspective_blit(image, image, transform)
cv2.imwrite('output.jpg', image)

示例结果:

这篇关于OpenCV warpPerspective-如何知道目标图像大小?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！