在Matlab中使用相关性进行模板匹配 [英] Template Matching using correlation in matlab

问题描述

我正在制作相机鼠标软件.处理实时视频以跟踪功能的移动并将其转换为鼠标的移动.

使用搜索窗口技术.使用相关的模板匹配是应用的技术.它是用matlab编码的.

发生的事情是该代码可以与预录制的视频一起正常运行，但不能实时运行.

知道为什么吗?

I am making a camera mouse software. Real time video is processed to track movement of the feature and translate to the mouse movement.

A search window technique is bing used. A template matching using correlation is the technique applied. Its coded in matlab.

What happens is the code works fine with a prerecorded video but not in real time.

Any idea why?

推荐答案

不要忘记，相关性取决于输入数据的当前状态和许多旧数据帧(取决于相关性大小) )以及记录的数据，您可以获得当前，过去和将来的所有数据.实时地，您仅具有当前数据，请确保您缓冲了足够的旧数据，以使关联过程能够正常工作.

don''t forget that correlation depends on the current state of the input data and a number of frames of old data (depending on correlation size), with recorded data, you have the present, past, and future data all available. in real-time you only have present data, make sure you have enough old data buffered to allow correlation process to work correctly.

< b> b</b>我不太明白你在说什么.
模板是静态的.不会改变.我们需要在后续框架中找到该模板.
它非常适合与预录的视频配合使用.
与相关公式本身有关吗?
我在这里附加代码.

trial2.m(用于获取要跟踪的功能.您将单击捕获并显示的图像.将保存一个围绕20 x 20的点的模板)

imaqreset;
clc;
清除；

vid = videoinput(''winvideo''，1，''YUY2_320x240'');

％配置参数
set(vid，''FramesPerTrigger''，3)
set(vid，''FrameGrabInterval''，3)
％vid.TriggerRepeat = inf;
set(vid，``LoggingMode''，``Memory'')
triggerconfig(vid，``手动'')

％设置内存
imaqmem(7000000000)
cnt = imaqmem;
mem_low = 1000000000;
mem_low1 = 2000000000;
mem_left = cnt.FrameMemoryLimit-cnt.FrameMemoryUsed;
start(vid);
％获取鼠标输入和触发器
预览(vid);
触发器(vid);
％end

frame = getsnapshot(vid);
frame = rgb2gray(frame);
delete(vid);

imshow(frame);
[inx iny] = ginput(1);
矩形(``位置''，[inx-10，iny-10,20,20]);
temp = frame(iny-10:iny + 10，inx-10:inx + 10);

％figure(1);
％imshow(temp); ％显示模板
r = eyeball2(temp，iny，inx);


||||||||||||||||||||||||||||||||||||||||||||||| |||||||||||||||||||||||||
eyeball2.m(捕获每帧并将其发送以用于模板匹配到mod3.m的功能)


函数r = eyeball2(template，x1，y1)
％重置图像获取工具
imaqreset;

r = -1;
vid = videoinput(''winvideo''，1，''YUY2_320x240'');

％配置参数
％set(vid，''FramesPerTrigger''，1)
％set(vid，''FrameGrabInterval''，1)
％vid.TriggerRepeat = inf;
％set(vid，``LoggingMode''，``Memory'')

triggerconfig(vid，``手动'');

％设置内存
imaqmem(700000000);
cnt = imaqmem;
mem_low = 10000000;
mem_low1 = 20000000;
mem_left = cnt.FrameMemoryLimit-cnt.FrameMemoryUsed;
start(vid);
％preview(vid);
％trigger(vid);
我= getsnapshot(vid);
i = 1;
while(mem_left> mem_low)
％while(i< = 20)
％trigger(vid);
我= getsnapshot(vid);
我= rgb2gray(I);

图(1);
％imshow(I);
imagesc(I);
％frame(:，:，i)= I;
等一下；
[a b] = mod3(I，template，x1，y1，i);
％放开；
x1 = a;
y1 = b;
i = i + 1;
％figure;
％imshow(template);
％if(mem_left< mem_low1)
％clear(frame);
％flushdata(vid);
％mem_left = cnt.FrameMemoryLimit-cnt.FrameMemoryUsed;
％i = 1;
％end
i = i + 1;
结束
停止(vid);



||||||||||||||||||||||||||||||||||||||||||||||| |||||||||||||||||||||||||
mod3.m(该函数从帧中提取蒙版，然后发送到correaltion函数，并相应地设置鼠标指针.需要安装Psychtoolbox才能使GetMouse()和SetMouse()函数起作用.


函数[x2，y2] = mod3(i1，i2，x1，y1，i)

％function
％参数
％i1:来自摄像头的帧
％i2:用于比较的模板
％x1:模板中心的先前x坐标
％y1:模板中心的上一个y坐标

％返回值
％x2:新模板中心的当前x坐标
％y2:新模板中心的当前y坐标
％x =行(最多240个)
％y = cols(最大320)

searchx = x1-50;
searchy = y1-50;
searchh = 100;
searchw = 100;

if(x1-50)< = 0
searchx = 1;
结束
if(x1 + 50)> = 240
searchx = 139;
结束
if(y1-50)< = 0
searchy = 1;
结束
if(y1 + 50)> = 320
searchy = 219;
结束

i1 = i1(searchx:searchx + searchh，searchy:searchy + searchw); ％i1现在具有搜索窗口
％figure(1);
％imshow(i1);

[r1 c1] = size(i1);
[r2 c2] = size(i2);
％corr = normxcorr2(i2，i1);
corr = zeros(r1-r2 + 1，c1-c2 + 1); ％相关矩阵

对于i = 1:r1-r2 + 1
对于j = 1:c1-c2 + 1
mask = i1(i:i + r2-1，j:j + c2-1);
corr(i，j)= correlationn(mask，i2);

结束
结束

查找最大百分比
[x y] = max(corr);
[x z] = max(x);
a1 = y(z);
b1 = z;
x％打印相关的最大值

％添加偏移量以获得相对于整个帧的坐标
x2 =(a1 + r2/2 + searchx);
y2 =(b1 + c2/2 + searchy);
％转换为整数，即整数
x2 = int32(x2);
y2 = int32(y2);
％set(gca，``Color''，[0，1，0]);
矩形(``位置''，[y2-8，x2-5，c2，r2]);

disx = abs(x2-x1);鼠标在x方向上的百分比绝对位移
disy = abs(y2-y1);鼠标在y方向上的百分比绝对位移

dispunit = 1;

％获取鼠标指针的当前位置
[mx my] = GetMouse();

if(disx< 5)
my = my;
elseif(x2> x1)
my = my + disx * dispunit;
elseif(x2< x1)
my = my- disx * dispunit;
其他
my = my;
结束

if(disy< 5)
mx = mx;
elseif y2> = y1
mx = mx + disy * dispunit;
elseif(y2< y1)
mx = mx- disy * dispunit;
其他
mx = mx;
结束
SetMouse(mx，my);


函数result = correlationn(s，t)
s = double(s);
t = double(t);
[m n] = size(s);
a = m * n;

mul = 0;
sums = sum(sum(s));
sumt = sum(sum(t));
sqs = sum(sum(s.^ 2));
sqt = sum(sum(s.^ 2));
mul = sum(sum(s.* t));
if(a * sqs-sums * sums< = 0)
结果= 0;
elseif(a * sqt-sumt * sumt< = 0)
结果= 0;
其他
sigs = sqrt(a * sqs-sums * sums);
sigt = sqrt(a * sqt-sumt * sumt);
结果=(a * mul-sums * sumt)/(sigs * sigt);
结束


||||||||||||||||||||||||||||||||||||||||||||||| |||||||||||||||||||||||||||||||||||

如果有人可以找到解决方案/错误，请通知我.谢谢.</b>

<b><b></b>i did not quite get what u are saying.
the template is static. does not change. we need to find that template in subsequent frames.
It works very precisely with a prerecorded video.
Is it to with the correlation formula itself?
Im attaching the code here.

trial2.m (for taking the feature to be tracked. you are to click on the image which is captured and displayed. a template around the point of 20 x 20 is saved)

imaqreset;
clc;
clear;

vid = videoinput(''winvideo'',1,''YUY2_320x240'');

% Configuring Parameter
set(vid,''FramesPerTrigger'',3)
set(vid,''FrameGrabInterval'',3)
%vid.TriggerRepeat = inf;
set(vid,''LoggingMode'',''Memory'')
triggerconfig(vid,''manual'')

% Set Memory
imaqmem(7000000000)
cnt = imaqmem;
mem_low = 1000000000;
mem_low1 = 2000000000;
mem_left = cnt.FrameMemoryLimit-cnt.FrameMemoryUsed;
start(vid);
%get mouse input and the trigger
preview(vid);
trigger(vid);
%end

frame = getsnapshot(vid);
frame=rgb2gray(frame);
delete(vid);

imshow(frame);
[inx iny]=ginput(1);
rectangle(''Position'',[inx-10,iny-10,20,20]);
temp=frame(iny-10:iny+10,inx-10:inx+10);

%figure(1);
%imshow(temp); %to disp template
r=eyeball2(temp,iny,inx);


|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
eyeball2.m (the function which captures each frame and send it for template matching to mod3.m)


function r= eyeball2(template,x1,y1)
%Resetting the image acquisition tool
imaqreset;

r=-1;
vid = videoinput(''winvideo'',1,''YUY2_320x240'');

% Configuring Parameter
%set(vid,''FramesPerTrigger'',1)
%set(vid,''FrameGrabInterval'',1)
%vid.TriggerRepeat = inf;
%set(vid,''LoggingMode'',''Memory'')

triggerconfig(vid,''manual'');

% Set Memory
imaqmem(700000000);
cnt = imaqmem;
mem_low = 10000000;
mem_low1 = 20000000;
mem_left = cnt.FrameMemoryLimit-cnt.FrameMemoryUsed;
start(vid);
%preview(vid);
%trigger(vid);
I = getsnapshot(vid);
i=1;
while(mem_left > mem_low)
%while(i<=20)
%trigger(vid);
I = getsnapshot(vid);
I = rgb2gray(I);

figure(1);
%imshow(I);
imagesc(I);
%frame(:,:,i)=I;
hold on;
[a b]=mod3(I,template,x1,y1,i);
%hold off;
x1=a;
y1=b;
i=i+1;
%figure;
%imshow(template);
% if (mem_left < mem_low1)
% clear(frame);
% flushdata(vid);
% mem_left = cnt.FrameMemoryLimit-cnt.FrameMemoryUsed;
% i=1;
% end
i=i+1;
end
stop(vid);



|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
mod3.m ( the function which extracts the mask from the frame and sends to the correaltion function and sets teh mouse pointer accordingly. Psychtoolbox needs to be installed for the GetMouse() and SetMouse() functions to work.


function [x2,y2]=mod3(i1,i2,x1,y1,i)

%function
%arguments
%i1: frame from webcam
%i2: template for comparison
%x1: previous x coordinates of centre of template
%y1: previous y coordinates of centre of template

%return values
%x2: current x coordinates of centre of new template
%y2: current y coordinates of centre of new template
%x=rows (max 240)
%y=cols (max 320)

searchx=x1-50;
searchy=y1-50;
searchh=100;
searchw=100;

if(x1-50)<=0
searchx=1;
end
if(x1+50)>=240
searchx=139;
end
if(y1-50)<=0
searchy=1;
end
if(y1+50)>=320
searchy=219;
end

i1=i1(searchx:searchx+searchh,searchy:searchy+searchw); %i1 now has the search window
%figure(1);
%imshow(i1);

[r1 c1]=size(i1);
[r2 c2]=size(i2);
% corr=normxcorr2(i2,i1);
corr=zeros(r1-r2+1,c1-c2+1); % correlation matrix

for i=1:r1-r2+1
for j=1:c1-c2+1
mask=i1(i:i+r2-1,j:j+c2-1);
corr(i,j)=correlationn(mask,i2);

end
end

%finding maximum
[x y]=max(corr);
[x z]=max(x);
a1=y(z);
b1=z;
x %printing maximum value of correlation

%adding offset to get coordinates relative to entire frame
x2=(a1+r2/2+searchx);
y2=(b1+c2/2+searchy);
%converting to integer i.e. whole number
x2=int32(x2);
y2=int32(y2);
%set(gca, ''Color'', [0, 1, 0]);
rectangle(''Position'',[y2-8,x2-5,c2,r2]);

disx=abs(x2-x1); %absolute displacement of mouse in x direction
disy=abs(y2-y1); %absolute displacement of mouse in y direction

dispunit=1;

%obtaining current location of mouse pointer
[mx my]=GetMouse();

if(disx<5)
my=my;
elseif(x2>x1)
my=my+ disx*dispunit;
elseif(x2<x1)
my=my- disx*dispunit;
else
my=my;
end

if(disy<5)
mx=mx;
elseif y2>=y1
mx=mx+disy*dispunit;
elseif(y2<y1)
mx=mx- disy*dispunit;
else
mx=mx;
end
SetMouse(mx,my);


function result=correlationn(s,t)
s=double(s);
t=double(t);
[m n]=size(s);
a=m*n;

mul=0;
sums=sum(sum(s));
sumt=sum(sum(t));
sqs=sum(sum(s.^2));
sqt=sum(sum(s.^2));
mul=sum(sum(s.*t));
if(a*sqs-sums*sums <=0)
result =0;
elseif(a*sqt-sumt*sumt<=0)
result=0;
else
sigs=sqrt(a*sqs-sums*sums);
sigt=sqrt(a*sqt-sumt*sumt);
result=(a*mul-sums*sumt)/(sigs*sigt);
end


|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

If anyone can find a solution/ the rror please let me know. Thank you.</b>

这篇关于在Matlab中使用相关性进行模板匹配的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！