在C ++/Obj-C ++中运行Tensorflow分类器模型的结果与Python不同 [英] Running Tensorflow Classifier Model in C++/Obj-C++ results in different result than Python

问题描述

我已经通过诗人的Tensorflow 重新训练了InceptionV3模型教程，并且可以成功地在我训练有素的数据和新数据上运行label_image.py，并获得具有正确性的正确标签.太棒了！

Ive retrained an InceptionV3 model via the Tensorflow for Poets tutorials and can successfully run label_image.py on my trained data and on new data and get correct labels with good accuracy. Awesome!

如果我通过Mac Obj-C ++应用程序运行模型，我得到的标签将完全不同.

If I run my model through my Mac Obj-C++ app my resulting labels are wildly different.

例如-我的训练是对视频帧的分类(极端特写，特写，中等，长，极长)进行分类，以对视频编辑内容进行分类.

For example - my training is to classify which 'shot type' a frame of video is, (extreme close up, close up, medium, long, extreme long) for classifying video editing content.

label_image.py将视频中的帧分类为85％可能会关闭. 我的C ++/Obj-C应用程序在相同框架下运行将其归类为60％的Extreme Long

label_image.py classifies a frame from a video as 85% likely close up. My C++ / Obj-C App run with the same frame classifies it as Extreme Long with 60%

两者均在通过AVX/SIMD/FMA优化编译的Mac OS X CPU上运行相同版本的Tensorflow(1.1).

Both are running the same version of Tensorflow (1.1) on Mac OS X CPU compiled with AVX/SIMD/FMA optimizations.

我的Apps管道:

我有一份BGR订购的OpenCV Mat映像，可以在其他地方成功使用它并从中获得合理的结果.我从映射到BGRA CV MAT的OS X CVPixelBufferRef创建CV Mat，如下所示:

I have a BGR ordered OpenCV Mat image which I can use successfully elsewhere and get sane results from. I create this CV Mat from an OS X CVPixelBufferRef mapped to a BGRA CV MAT like so:

cv::cvtColor(BGRAImage, frameMat, cv::COLOR_BGRA2BGR);

我通过从iOS contrib示例中借来的代码将BGR CV Mat(命名为frameMat)输入张量，就像这样:

I feed that BGR CV Mat (named frameMat) into a Tensor via code borrowed from the iOS contrib example, like so :

 void* baseAddress = (void*)frameMat.datastart;
 size_t width = (size_t) frameMat.cols;
 size_t height = (size_t) frameMat.rows;
 size_t bytesPerRow =  (size_t) frameMat.cols * 3; // (BGR)

 const int wanted_input_width = 299;
 const int wanted_input_height = 299;
 const int wanted_input_channels = 3;
 const float input_mean = 128.0f;
 const float input_std = 128.0f;

 resized_tensor = tensorflow::Tensor( tensorflow::DT_FLOAT, tensorflow::TensorShape({1, wanted_input_height, wanted_input_width, wanted_input_channels}));

auto image_tensor_mapped = resized_tensor.tensor<float, 4>();
tensorflow::uint8 *in = sourceStartAddr;
float *out = image_tensor_mapped.data();
for (int y = 0; y < wanted_input_height; ++y)
{
    float *out_row = out + (y * wanted_input_width * wanted_input_channels);
    for (int x = 0; x < wanted_input_width; ++x)
    {
        const int in_x = (y * (int)width) / wanted_input_width;
        const int in_y = (x * image_height) / wanted_input_height;

        tensorflow::uint8 *in_pixel = in + (in_y * width * (image_channels)) + (in_x * (image_channels));
        float *out_pixel = out_row + (x * wanted_input_channels);

        // Interestingly the iOS example uses BGRA and DOES NOT re-order tensor channels to RGB <-> BGR
        // Matching that.
        out_pixel[0] = ((float)in_pixel[0] - (float)input_mean) / (float)input_std;
        out_pixel[1] = ((float)in_pixel[1] - (float)input_mean) / (float)input_std;
        out_pixel[2] = ((float)in_pixel[2] - (float)input_mean) / (float)input_std;
    }
}

我的会话创建代码:

 tensorflow::Status load_graph_status = ReadBinaryProto(tensorflow::Env::Default(), [inception2015GraphPath cStringUsingEncoding:NSUTF8StringEncoding], &inceptionGraphDef);

 if (load_graph_status.ok())
 {      
      tensorflow::SessionOptions options;        
      inceptionSession = std::unique_ptr<tensorflow::Session>(tensorflow::NewSession(options));        
      tensorflow::Status session_create_status = inceptionSession->Create(inceptionGraphDef);
 }

运行图形:

    tensorflow::Status run_status = inceptionSession->Run({ {input_layer, resized_tensor} }, {feature_layer, final_layer}, {}, &outputs);

并拉出标签/特征向量(倒数第二层)

And pulling out the labels / feature vector (penultimate layer)

    NSMutableArray* outputLabels = [NSMutableArray arrayWithCapacity:self.labelsArray.count];
NSMutableArray* outputScores = [NSMutableArray arrayWithCapacity:self.labelsArray.count];

// 1 = labels and scores
auto predictions = outputs[1].flat<float>();

for (int index = 0; index < predictions.size(); index += 1)
{
    const float predictionValue = predictions(index);

    NSString* labelKey  = self.labelsArray[index % predictions.size()];

    NSNumber* currentLabelScore = self.averageLabelScores[labelKey];

    NSNumber* incrementedScore = @([currentLabelScore floatValue] + predictionValue );
    self.averageLabelScores[labelKey] = incrementedScore;

    [outputLabels addObject:labelKey];
    [outputScores addObject:@(predictionValue)];
}

// 0 is feature vector
tensorflow::Tensor feature = outputs[0];
int64_t numElements = feature.NumElements();
tensorflow::TTypes<float>::Flat featureVec = feature.flat<float>();

NSMutableArray* featureElements = [NSMutableArray arrayWithCapacity:numElements];

for(int i = 0; i < numElements; i++)
{
    [featureElements addObject:@( featureVec(i) ) ];
}

if(self.averageFeatureVec == nil)
{
    self.averageFeatureVec = featureElements;
}
else
{
    // average each vector element with the prior
    for(int i = 0; i < featureElements.count; i++)
    {
        float  a = [featureElements[i] floatValue];
        float  b = [self.averageFeatureVec[i] floatValue];

        self.averageFeatureVec[i] = @( MAX(a,b)) ;
    }
}

return @{ kSynopsisStandardMetadataFeatureVectorDictKey : featureElements ,
          @"Labels" : outputLabels,
          @"Scores" : outputScores,
          };

我试图研究张量排序(NHWC)，并检查了张量创建代码，但我可能会遗漏一些对其他人显而易见的东西. Ive也尝试更改频道顺序，但无济于事.

I've attempted to look into the tensor ordering (NHWC), and have checked the tensor creation code but I might be missing something obvious to others. Ive also tried changing channel order, to no avail.

任何见解将大有帮助.谢谢！

Any insight would be greatly helpful. Thank you!

推荐答案

所以这很棘手.

我没有提到我在重新训练的图形上运行graph_transform工具-并且正在运行量化权重以减小图形尺寸.过去，我完全没有搞乱分类得分的问题，但是显然这引起了问题.

I failed to mention I was running the graph_transform tool on my retrained graph - and was running quantize weights to lower my graphs size. In the past, I've not had an issue with this messing up classification scores at all, but apparently that caused an issue.

在没有量化权重的情况下通过图形变换调用运行上述代码，从而解决了此问题.

Running the above code with a graph transform call without quantize weights fixed the issue.

这篇关于在C ++/Obj-C ++中运行Tensorflow分类器模型的结果与Python不同的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！