对包含折线图的两个图像进行相似性度量的好方法是什么? [英] What is a good way to get a similarity measure of two images that contain a line chart?

问题描述

我尝试了将dHash算法应用于每个图像，然后在两个哈希值上计算hamming_distance，数字越小，相似度越高.

I have tried the dHash algorithm which is applied on each image, then a hamming_distance is calculated on both hashes, the lower the number, the higher the similarity.

from PIL import Image
import os
import shutil
import glob
from plotData import *

def hamming_distance(s1, s2):
    #Return the Hamming distance between equal-length sequences
    if len(s1) != len(s2):
        raise ValueError("Undefined for sequences of unequal length")
    return sum(ch1 != ch2 for ch1, ch2 in zip(s1, s2))


def dhash(image, hash_size = 8):
    # Grayscale and shrink the image in one step.
    image = image.convert('L').resize(
        (hash_size + 1, hash_size),
        Image.ANTIALIAS,
    )

    pixels = list(image.getdata())

    # Compare adjacent pixels.
    difference = []
    for row in xrange(hash_size):
        for col in xrange(hash_size):
            pixel_left = image.getpixel((col, row))
            pixel_right = image.getpixel((col + 1, row))
            difference.append(pixel_left > pixel_right)

    # Convert the binary array to a hexadecimal string.
    decimal_value = 0
    hex_string = []
    for index, value in enumerate(difference):
        if value:
            decimal_value += 2**(index % 8)
        if (index % 8) == 7:
            hex_string.append(hex(decimal_value)[2:].rjust(2, '0'))
            decimal_value = 0

    return ''.join(hex_string)




orig = Image.open('imageA.png')
modif = Image.open('imageA.png')
hammingDistanceValue = hamming_distance(dhash(orig),dhash(modif))
    print hammingDistanceValue

不幸的是，这种方法会产生误报，因为它并未真正将折线图形状视为主要相似特征.我猜想，我需要某种机器学习方法，也许是来自openCV的方法.谁能指导我朝着正确的方向发展，以达到高精确度?

Unfortunately, this approach produces false positives because it does not really look at the line chart shapes as primary similarity feature. I guess, I'd need some kind of machine learning approach maybe from openCV or so. Can anyone guide me into the right direction to something that compares with high precision?

这是要与一组相似图像进行比较的初始图像.

这是积极匹配

this is a positive match

这是 false 匹配

this is a false match

更新:我在下面的jme建议中添加了一些opencv魔术.我尝试先检测重要功能.但是，它仍然会产生误报，因为相似性的总体指标是所有要素的累加值，并且没有考虑差异，因此可以赋予折线图完全不同的含义.

update: I added some opencv magic to jme's suggestion below. I try to detect significant features first. Howeve, it still produces false positives, since the overall indicator for similarity is the cummulated value over all features and does not take differences into account that can give a line chart a totally different meaning.

假阳性示例

具有明显特征标记为红点的预处理图像示例

from PIL import Image
import os
import numpy as np
from scipy.interpolate import interp1d
import os.path
import shutil
import glob
from plotData import *
import cv2
from matplotlib import pyplot as plt

def load_image(path):
    #data = Image.open(path)
    img = cv2.imread(path)
    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

    corners = cv2.goodFeaturesToTrack(gray,25,0.01,10)
    corners = np.int0(corners)

    for i in corners:
        x,y = i.ravel()
        cv2.circle(img,(x,y),3,255,-1)

    return np.mean((255 - np.array(img))**2, axis=2)


symbol = "PBYI"
x = np.arange(1000)

if not os.path.exists('clusters1DSignal/'+symbol+'/'):
    os.mkdir('clusters1DSignal/'+symbol+'/')
else:
    shutil.rmtree('clusters1DSignal/'+symbol+'/')
    os.mkdir('clusters1DSignal/'+symbol+'/')

shutil.copyfile('rendered/'+symbol+'.png', "clusters1DSignal/"+symbol+"/"+symbol+'.png')


img1 = load_image('rendered/'+symbol+'.png')
y1 = np.argmax(img1, axis=0)
f1 = interp1d(np.linspace(0, 1000, len(y1)), y1)
z1 = f1(x)

for filename in glob.iglob('rendered/*.png'):
    try:
        img2 = load_image(filename)
    except:
        continue
    y2 = np.argmax(img2, axis=0)
    f2 = interp1d(np.linspace(0, 1000, len(y2)), y2)
    z2 = f2(x)

    result = np.linalg.norm(z1 - z2)
    if result < 2100:
        print str(result) +": " +filename
        symbolCompare = filename.split("/")[1].replace(".png","")
        shutil.copyfile('rendered/'+symbolCompare+'.png', "clusters1DSignal/"+symbol+"/"+str(result)+"_"+symbolCompare+".png")

推荐答案

我要采取的方法是:首先，通过为每个x像素找到一个代表性的y像素，将每个图像转换为1d信号.图像为红色的位置.您可以取y像素的平均值，但为简单起见，我将取第一个不是白色的像素:

The approach I'd take is this: first, convert each image to a 1d signal by finding for each x pixel, a representative y pixel where the image is red. You can take the mean of the y pixels, but for simplicity, I'll just take the first that isn't white:

def load_image(path):
    data = Image.open(path)
    return np.mean((255 - np.array(data))**2, axis=2)

img1 = load_image("one.png")
img2 = load_image("two.png")
img3 = load_image("three.png")

y1 = np.argmax(img1, axis=0)
y2 = np.argmax(img2, axis=0)
y3 = np.argmax(img3, axis=0)

y1，y2和y3是一维数组，表示第一张，第二张和第三张图像中的函数.现在，我们将每个数组简单地视为一个向量，并找到它们之间的l2距离.我们更喜欢l2距离，因为汉明距离对于此任务会有些敏感.

y1, y2, and y3 are 1d arrays which represent the functions in the first, second, and third images. Now we simply treat each array as a vector, and find the l2 distance between them. We prefer the l2 distance because the Hamming distance will be somewhat sensitive for this task.

我们有一个小问题:图像的宽度不同，因此y数组的大小不兼容.一种快速而又肮脏的解决方法是将它们插值更长的长度(我们将使用1000):

We have a slight problem: the images have different widths, so the y arrays aren't of compatible size. A quick-and-dirty fix is to interpolate them to a longer length (we'll use 1000):

f1 = interp1d(np.linspace(0, 1000, len(y1)), y1)
f2 = interp1d(np.linspace(0, 1000, len(y2)), y2)
f3 = interp1d(np.linspace(0, 1000, len(y3)), y3)

x = np.arange(1000)
z1 = f1(x)
z2 = f2(x)
z3 = f3(x)

现在我们可以找到图像之间的距离:

Now we can find the distance between the images:

>>> np.linalg.norm(z1 - z2)
2608.5368359281415
>>> np.linalg.norm(z1 - z3)
5071.1340610709549
>>> np.linalg.norm(z2 - z2)
5397.379183811714

这篇关于对包含折线图的两个图像进行相似性度量的好方法是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！