跟踪视频中的眼睛瞳孔 [英] track eye pupil in a video

问题描述

我正在开展一项旨在跟踪眼球学生的项目。为此，我制作了一个头戴式系统，捕捉眼睛的图像。完成硬件部分我在软件部分。我使用 opencv 。请让我知道什么是跟踪学生的最有效的方法。 Houghcircles 效果不佳。

I am working on a project aimed to track eye pupil. For this I have made a head-mounted system that captures the images of the eye. Completed with the hardware portion I am struck in software part. I am using opencv. Please let me know what would be the most efficient way to track the pupil. Houghcircles didn't performing well.

之后，我也尝试过 HSV 过滤器，这里是代码和
链接到原始图像的屏幕截图，处理一个。请帮助我解决此问题。该链接还包含我在此代码中使用的眼睛学生的视频。

After that I have also tried with HSV filter and here is the code and link to screenshot of the raw-image and processed one. Please help me to resolve this issue. The link also contains video of eye pupil that I am using in this code.

https://picasaweb.google.com/118169326982637604860/16November2011?authuser=0&authkey=Gv1sRgCPKwwrGTyvX1Aw&feat=directlink

代码：

include "cv.h"

include"highgui.h"

IplImage* GetThresholdedImage(IplImage* img)
{

    IplImage *imgHSV=cvCreateImage(cvGetSize(img),8,3);
    cvCvtColor(img,imgHSV,CV_BGR2HSV);
    IplImage *imgThresh=cvCreateImage(cvGetSize(img),8,1);
    cvInRangeS(imgHSV,cvScalar(0, 84, 0, 0),cvScalar(179, 256, 11, 0),imgThresh);
    cvReleaseImage(&imgHSV);
    return imgThresh;
}

void main(int *argv,char **argc)
{

    IplImage *imgScribble= NULL;
    char c=0;
    CvCapture *capture;
    capture=cvCreateFileCapture("main.avi");

    if(!capture)
    {
        printf("Camera could not be initialized");
        exit(0);
    }
    cvNamedWindow("Simple");
    cvNamedWindow("Thresholded");

    while(c!=32)
    {
        IplImage *img=0;
        img=cvQueryFrame(capture);
        if(!img)
            break;
        if(imgScribble==NULL)
            imgScribble=cvCreateImage(cvGetSize(img),8,3);

        IplImage *timg=GetThresholdedImage(img);
        CvMoments *moments=(CvMoments*)malloc(sizeof(CvMoments));
        cvMoments(timg,moments,1);

        double moment10 = cvGetSpatialMoment(moments, 1, 0);
        double moment01 = cvGetSpatialMoment(moments, 0, 1);
        double area = cvGetCentralMoment(moments, 0, 0);

        static int posX = 0;
        static int posY = 0;

        int lastX = posX;
        int lastY = posY;

        posX = moment10/area;
        posY = moment01/area;
         // Print it out for debugging purposes
        printf("position (%d,%d)\n", posX, posY);
        // We want to draw a line only if its a valid position
        if(lastX>0 && lastY>0 && posX>0 && posY>0)
        {
            // Draw a yellow line from the previous point to the current point
            cvLine(imgScribble, cvPoint(posX, posY), cvPoint(lastX, lastY), cvScalar(0,255,255), 5);
        }
        // Add the scribbling image and the frame...

        cvAdd(img, imgScribble, img);

        cvShowImage("Simple",img);
        cvShowImage("Thresholded",timg);
        c=cvWaitKey(3);
        cvReleaseImage(&timg);
        delete moments;

    }
    //cvReleaseImage(&img);
    cvDestroyWindow("Simple");
    cvDestroyWindow("Thresholded");

}

---

能够跟踪眼睛并精确地找到瞳孔的中心坐标。

---

I am able to track the eye and find the center coordinates of pupil precisely.

首先我对头戴式相机拍摄的图像进行阈值处理。之后，我使用轮廓查找算法，然后我找到所有轮廓的质心。这给出了眼睛瞳孔的中心坐标，该方法实时地工作良好，并且还以非常好的精度检测眨眼。

First I thresholded the image taken by the head mounted camera. After that I have used contour finding algorithm then I find the centroid of all the contours. This gives me the center coordinates of eye pupil, this method is working fine in real time and also detecting eye blinking with very good accuracy.

现在，我的目标是将这个功能嵌入到游戏（赛车游戏）中。其中，如果我看左/右，那么汽车左/右移动，如果我眨眼车慢下来。我该如何继续？我需要一个游戏引擎来做到这一点吗？

Now, my aim is to embed this feature into a game(a racing game). In which If I look to left/right then the car moves left/right and If I blink the car slows down. How could I proceed now??? Would I need a game engine to do that?

我听说过一些与visual studio 2010（unity等）兼容的开源游戏引擎。是可行吗？如果是，我应该如何进行？

I heard of some open source game engines compatible with visual studio 2010(unity etc.). Is it feasible??? If yes, how should I proceed ?

推荐答案

我是SimpleCV的开发人员之一。我们维护一个用于计算机视觉的开源python库。您可以在 SimpleCV.org 下载。 SimpleCV非常适合通过在命令行上进行黑客攻击来解决这些类型的问题。我只能在几行代码中提取瞳孔。这里你去：

I am one of the developers of SimpleCV. We maintain an open-source python library for computer vision. You can download it at SimpleCV.org. SimpleCV is great for solving these types of problems by hacking on the command line. I was able to extract the pupil in only a couple lines of code. Here you go:

img = Image("eye4.jpg") # load the image
bm = BlobMaker() # create the blob extractor
# invert the image so the pupil is white, threshold the image, and invert again
# and then extract the information from the image
blobs = bm.extractFromBinary(img.invert().binarize(thresh=240).invert(),img)

if(len(blobs)>0): # if we got a blob
    blobs[0].draw() # the zeroth blob is the largest blob - draw it
    locationStr = "("+str(blobs[0].x)+","+str(blobs[0].y)+")"
    # write the blob's centroid to the image
    img.dl().text(locationStr,(0,0),color=Color.RED)
    # save the image
    img.save("eye4pupil.png")
    # and show us the result.
    img.show()

以下是结果。

Here are the results.

因此，接下来的步骤是使用某种跟踪器，如卡尔曼滤波器，跟踪瞳孔。您可能想要将眼睛建模为球体并在球面坐标（即θ和phi）中跟踪瞳孔的质心。你还需要写一点代码来检测眨眼事件，这样当用户眨眼时系统不会变得。。。我建议使用canny边缘检测器找到图像中最大的水平线，并假设这些是眼睑。我希望这有助于，请让我们知道你的工作进展。

So your next steps are to use some sort of tracker, like a Kalmann filter, to track the pupil robustly. You may want to model the eye as a sphere and track the pupil's centroid in sphereical coordinates (i.e. theta and phi). You will also want to write a bit of code to detect blink events so the system doesn't go all wonky when the user blinks. I suggest using a canny edge detector to find the largest horizontal lines in the image and assuming those are the eye lids. I hope this helps and please let us know how your work progresses.

这篇关于跟踪视频中的眼睛瞳孔的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！