如何在Dlib C ++中获得3D头部姿势估计坐标轴 [英] How to get 3D coordinate Axes of head pose estimation in Dlib C++

问题描述

Dlib C ++可以很好地检测界标并估计人脸姿势.但是，如何获得头部姿势的3D坐标轴方向(x，y，z)?

Dlib C++ can detect landmark and estimate face pose very well. However, how can I get 3D coordinate Axes direction (x,y,z) of head pose?

推荐答案

不久前，我也面临着同样的问题，搜索并找到了1-2条有用的博客文章，这是 github 页面提取的最小的工作代码段看起来像这样:

I was also facing the same issue, a while back ago, searched and found 1-2 useful blog posts, this link would get you an overview of the techniques involved, If you only need to calculate the 3D pose in decimal places then you may skip the OpenGL rendering part, However if you want to visually get the Feedback then you may try with OpenGL as well, But I would suggest you to ignore the OpenGL part as a beginner, So the smallest working code snippet extracted from github page, would look something like this:

// Reading image using OpenCV, you may use dlib as well.
cv::Mat img = cv::imread(imagePath);

std::vector<double> rv(3), tv(3);
cv::Mat rvec(rv),tvec(tv);
cv::Vec3d eav;

// Labelling the 3D Points derived from a 3D model of human face.
// You may replace these points as per your custom 3D head model if any
std::vector<cv::Point3f > modelPoints;
modelPoints.push_back(cv::Point3f(2.37427,110.322,21.7776));    // l eye (v 314)
modelPoints.push_back(cv::Point3f(70.0602,109.898,20.8234));    // r eye (v 0)
modelPoints.push_back(cv::Point3f(36.8301,78.3185,52.0345));    //nose (v 1879)
modelPoints.push_back(cv::Point3f(14.8498,51.0115,30.2378));    // l mouth (v 1502)
modelPoints.push_back(cv::Point3f(58.1825,51.0115,29.6224));    // r mouth (v 695)
modelPoints.push_back(cv::Point3f(-61.8886f,127.797,-89.4523f));  // l ear (v 2011)
modelPoints.push_back(cv::Point3f(127.603,126.9,-83.9129f));     // r ear (v 1138)

// labelling the position of corresponding feature points on the input image.
std::vector<cv::Point2f> srcImagePoints = {cv::Point2f(442, 442), // left eye
                                           cv::Point2f(529, 426), // right eye
                                           cv::Point2f(501, 479), // nose
                                           cv::Point2f(469, 534), //left lip corner
                                           cv::Point2f(538, 521), // right lip corner
                                           cv::Point2f(349, 457), // left ear
                                           cv::Point2f(578, 415) // right ear};


cv::Mat ip(srcImagePoints);

cv::Mat op = cv::Mat(modelPoints);
cv::Scalar m = mean(cv::Mat(modelPoints));

rvec = cv::Mat(rv);
double _d[9] = {1,0,0,
                0,-1,0,
                0,0,-1};
Rodrigues(cv::Mat(3,3,CV_64FC1,_d),rvec);
tv[0]=0;tv[1]=0;tv[2]=1;
tvec = cv::Mat(tv);


double max_d = MAX(img.rows,img.cols);
double _cm[9] = {max_d,     0, (double)img.cols/2.0,
                 0    , max_d, (double)img.rows/2.0,
                 0    ,     0,                  1.0};
cv::Mat camMatrix = cv::Mat(3,3,CV_64FC1, _cm);

double _dc[] = {0,0,0,0};
solvePnP(op,ip,camMatrix,cv::Mat(1,4,CV_64FC1,_dc),rvec,tvec,false,CV_EPNP);

double rot[9] = {0};
cv::Mat rotM(3,3,CV_64FC1,rot);
Rodrigues(rvec,rotM);
double* _r = rotM.ptr<double>();
printf("rotation mat: \n %.3f %.3f %.3f\n%.3f %.3f %.3f\n%.3f %.3f %.3f\n",
       _r[0],_r[1],_r[2],_r[3],_r[4],_r[5],_r[6],_r[7],_r[8]);

printf("trans vec: \n %.3f %.3f %.3f\n",tv[0],tv[1],tv[2]);

double _pm[12] = {_r[0],_r[1],_r[2],tv[0],
                  _r[3],_r[4],_r[5],tv[1],
                  _r[6],_r[7],_r[8],tv[2]};

cv::Mat tmp,tmp1,tmp2,tmp3,tmp4,tmp5;
cv::decomposeProjectionMatrix(cv::Mat(3,4,CV_64FC1,_pm),tmp,tmp1,tmp2,tmp3,tmp4,tmp5,eav);
printf("Face Rotation Angle:  %.5f %.5f %.5f\n",eav[0],eav[1],eav[2]);

输出:

                       **X**     **Y**    **Z**

Face Rotation Angle:  171.44027 -8.72583 -9.90596

这篇关于如何在Dlib C ++中获得3D头部姿势估计坐标轴的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！