如何使用变形网格扭曲图像 [英] How to warp an image using deformed mesh

问题描述

我正在尝试生成皱巴巴的"图像使用从平板扫描仪获得的图像.

按照论文

这是说明合成图像生成的论文截图:

用于测试的示例源图像:

(4) 裁剪非零区域并在必要时调整大小:

---

相关:

1. 从 c++ 转换 opencv 重映射代码到蟒蛇

2. 在扫描文档中拆分文本行

I'm trying to generate "crumpled" images using images obtained from a flatbed bed scanner.

By following the method described in the paper[Link] in the section 3.1. I've written the code generate the perturbed mesh but I don't know how to map this pixels from source image onto this mesh to form the perturbed image.

This is the code to generate the perturbed mesh.

import numpy as np
import matplotlib.pyplot as plt

mr = 88
mc = 68

xx = np.arange(mr-1, -1, -1)
yy = np.arange(0, mc, 1)
[Y, X] = np.meshgrid(xx, yy)
ms = np.transpose(np.asarray([X.flatten('F'), Y.flatten('F')]), (1,0))

perturbed_mesh = ms
nv = np.random.randint(20) - 1
for k in range(nv):
    #Choosing one vertex randomly
    vidx = np.random.randint(np.shape(ms)[0])
    vtex = ms[vidx, :]
    #Vector between all vertices and the selected one
    xv  = perturbed_mesh - vtex
    #Random movement 
    mv = (np.random.rand(1,2) - 0.5)*20
    hxv = np.zeros((np.shape(xv)[0], np.shape(xv)[1] +1) )
    hxv[:, :-1] = xv
    hmv = np.tile(np.append(mv, 0), (np.shape(xv)[0],1))
    d = np.cross(hxv, hmv)
    d = np.absolute(d[:, 2])
    d = d / (np.linalg.norm(mv, ord=2))
    wt = d
    
    curve_type = np.random.rand(1)
    if curve_type > 0.3:
        alpha = np.random.rand(1) * 50 + 50
        wt = alpha / (wt + alpha)
    else:
        alpha = np.random.rand(1) + 1
        wt = 1 - (wt / 100 )**alpha
    msmv = mv * np.expand_dims(wt, axis=1)
    perturbed_mesh = perturbed_mesh + msmv

plt.scatter(perturbed_mesh[:, 0], perturbed_mesh[:, 1], c=np.arange(0, mr*mc))
plt.show()

This is how the perturbed mesh looks like:

This is the screenshot from the paper illustrating the synthetic image generation:

Sample source image for testing: https://i.stack.imgur.com/26KN4.jpg

I'm stuck with mapping the source image pixels onto the mesh. I'll be grateful if someone can help.

解决方案

(1) Use cv2.copyMakeBorder to enlarge the image, to avoid the warpped points going out of range of the original image size.

cv2.copyMakeBorder(...)
    copyMakeBorder(src, top, bottom, left, right, borderType[, dst[, value]]) -> dst
    .   @brief Forms a border around an image.
    .
    .   The function copies the source image into the middle of the destination image. The areas to the
    .   left, to the right, above and below the copied source image will be filled with extrapolated
    .   pixels. This is not what filtering functions based on it do (they extrapolate pixels on-fly), but
    .   what other more complex functions, including your own, may do to simplify image boundary handling.

useage:

img = cv2.copyMakeBorder(img, dh, dh, dw, dw, borderType=cv2.BORDER_CONSTANT, value=(0,0,0))

Set dw=nw//2, dh=nh//2 maybe ok, adjust if necessary. The nh, nw is the height and width of the source image.

(2) Create perturbed mesh grid using the method from the paper

xs, ys = create_grid() # the result is like np.meshgrid

Notice make sure the type and the size.

# xs = xs.reshape(nh, nw).astype(np.float32)
# nh, nw is the height and width of the coppied image

(3) Use cv2.remap to remap:

cv2.remap(...)
    remap(src, map1, map2, interpolation[, dst[, borderMode[, borderValue]]]) -> dst
    .   @brief Applies a generic geometrical transformation to an image.
    .
    .   The function remap transforms the source image using the specified map:
    .   f[	exttt{dst} (x,y) =  	exttt{src} (map_x(x,y),map_y(x,y))f]

usage:

dst= cv2.remap(img, xs, ys, cv2.INTER_CUBIC)

This is a demo result:

(4) Crop the nonzero region and resize if necessary:

---

Related:

1. Converting opencv remap code from c++ to python

2. Split text lines in scanned document

这篇关于如何使用变形网格扭曲图像的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！