重新训练MobileNet SSD V1 COCO后，Tensorflow的pb和pbtxt文件无法与OpenCV一起使用 [英] Tensorflow' pb and pbtxt files don't work with OpenCV after retraining MobileNet SSD V1 COCO
问题描述
我已遵循此教程来重新培训MobileNet SSD V1如前所述，使用Tensorflow GPU并在使用GPU (有关配置的更多信息)训练后得到0.5的损失，并得到model.ckpt.
I have followed this tutorial to retrain MobileNet SSD V1 using Tensorflow GPU as described and got 0.5 loss after training using GPU (below more info about config) and got model.ckpt.
这是我用于训练的命令:
This is the command I used for Training:
python ../models/research/object_detection/legacy/train.py --logtostderr --train_dir=./data/ --pipeline_config_path=./ssd_mobilenet_v1_pets.config
这是冻结命令(生成pb文件):
And this is the command for freezing (generate pb file):
python ../models/research/object_detection/export_inference_graph.py --input_type image_tensor --pipeline_config_path ./ssd_mobilenet_v1_pets.config --trained_checkpoint_prefix ./data/model.ckpt-1407 --output_directory ./data/
这是我使用冻结的pb和pbtxt时遇到的错误:
This is the error I get when I use frozen pb and pbtxt:
Traceback (most recent call last):
File "Object_detection_image.py", line 29, in <module>
cvOut = cvNet.forward()
cv2.error: OpenCV(3.4.3) C:\projects\opencv-python\opencv\modules\dnn\src\dnn.cpp:565: error: (-215:Assertion failed) inputs.size() == requiredOutputs in function 'cv::dnn::experimental_dnn_34_v7::DataLayer::getMemoryShapes'
这是我使用的Object_detection_image.py文件:
import cv2 as cv
import os
import time
import logging
logger = logging.getLogger()
fh = logging.FileHandler('xyz.log')
fh.setLevel(logging.DEBUG)
logger.addHandler(fh)
cvNet = cv.dnn.readNetFromTensorflow('frozen_inference_graph.pb', 'object_detection.pbtxt')
dir_x = "C:\\Users\\Omen\\Desktop\\LP_dataset\\anno"
for filename in os.listdir(dir_x):
print(filename)
if not (filename.endswith(".png") or filename.endswith(".jpg")):
continue
print('daz')
img = cv.imread(os.path.join(dir_x,filename))
img = cv.resize(img, (300,300))
#cv.imshow('i',img)
#cv.waitKey(0)
img = cv.cvtColor(img,cv.COLOR_BGR2GRAY)
img = cv.cvtColor(img,cv.COLOR_GRAY2RGB)
rows = img.shape[0]
cols = img.shape[1]
#cvNet.setInput(cv.dnn.blobFromImage(img, size=(cols,rows), swapRB=True, crop=False))
cvNet.setInput(cv.dnn.blobFromImage(img, size=(300, 300), crop=False))
t0 = time.time()
cvOut = cvNet.forward()
print(time.time() - t0)
for detection in cvOut[0,0,:,:]:
score = float(detection[2])
#print(score)
if score > 0.80:
left = detection[3] * cols
top = detection[4] * rows
right = detection[5] * cols
bottom = detection[6] * rows
cv.rectangle(img, (int(left), int(top)), (int(right), int(bottom)), (23, 230, 210), thickness=2)
cv.imshow('img', img)
cv.waitKey(0)
这是pbtxt文件(我也尝试了导出的pbtxt并从pb生成了pbtxt，但无法正常工作):
item {
id: 1
name: 'licenseplate'
}
配置:
您正在使用的模型的顶级目录是什么:object_detetion
What is the top-level directory of the model you are using: object_detetion
我是否编写了自定义代码:否
Have I written custom code: no
OS平台和发行版:win10
OS Platform and Distribution: win10
TensorFlow安装于:二进制
TensorFlow installed from: binary
TensorFlow GPU版本:1.13.0
TensorFlow GPU version: 1.13.0
CUDA/cuDNN版本:10
CUDA/cuDNN version: 10
GPU型号:1050 GTX
GPU model: 1050 GTX
我可以提供您要求的任何文件，请帮助我. 在tensorflow的github中，他们告诉我在Stackoverflow中问...
I can provide any files you ask, please help me. In tensorflow's github they told me to ask in Stackoverflow...
更新:
我的问题得到了解决，这是cvOut的内容:
I got the problem solved thanks to the answer, here is the content of cvOut:
[[[[-0.00476191 -0.00361736 0. 0.25361738 -0.07576995
0.03405379 0.40910327]
[ 0.21594621 0.04544836 0. 0.28788495 0.30689242
-0.13025634 0.05074273]
[ 0.46358964 0.19925728 0. -0.09778295 0.26563603
0.34778297 -0.02014329]
[-0.01515752 0.3534766 0. 0.32857144 -0.00361736
0.67142856 0.25361738]
[ 0.25756338 0.03405379 0. 0.21594621 0.3787817
-0.05689242 0.6212183 ]
[ 0.30689242 0.203077 0. 0.796923 0.19925728
0.40103063 -0.09778295]
[ 0.5989694 0.34778297 0. -0.01515752 0.68680996
0.26515752 0.66190475]
[-0.00361736 1.0047619 0. 0.59089667 0.03405379
1.0757699 0.21594621]
[ 0.712115 -0.05689242 0. 0.30689242 0.53641033
0.05074273 1.1302563 ]
[ 0.19925728 0.7343639 0. 0.93230265 0.34778297
0.64652336 -0.01515752]
[ 1.0201433 0.26515752 0. 0.24638264 0.33809522
0.50361735 -0.07576995]
[ 0.2840538 0.40910327 0. 0.04544836 0.19310758
0.28788495 0.5568924 ]
[-0.13025634 0.30074272 0. 0.44925728 0.06769729
0.15221705 0.26563603]
[ 0.59778297 -0.02014329 0. 0.3534766 0.5151575
0.32857144 0.24638264]
[ 0.67142856 0.50361735 0. 0.2840538 0.7424366
0.4659462 0.3787817 ]
[ 0.19310758 0.6212183 0. 0.203077 0.30074272
0.796923 0.44925728]
[ 0.40103063 0.15221705 0. 0.59778297 0.31319004
0.23484248 0.68680996]
[ 0.5151575 0.66190475 0. 1.0047619 0.50361735
0.59089667 0.2840538 ]
[ 1.0757699 0.4659462 0. 0.19310758 0.95455164
0.5568924 0.53641033]
[ 0.30074272 1.1302563 0. 0.7343639 0.15221705
0.93230265 0.59778297]
[ 0.64652336 0.23484248 0. 0.5151575 -0.00476191
0.49638262 0.33809522]
[ 0.75361735 -0.07576995 0. 0.40910327 0.7159462
0.04544836 0.44310758]
[ 0.28788495 0.8068924 0. 0.55074275 0.46358964
0.69925725 0.06769729]
[ 0.40221703 0.26563603 0. -0.02014329 0.48484248
0.3534766 0.7651575 ]
[ 0.32857144 0.49638262 0. 0.75361735 0.25756338
0.5340538 0.7424366 ]
[ 0.7159462 0.3787817 0. 0.6212183 0.8068924
0.203077 0.55074275]
[ 0.796923 0.69925725 0. 0.40221703 0.5989694
0.84778297 0.31319004]
[ 0.48484248 0.68680996 0. 0.66190475 0.49638262
1.0047619 0.75361735]
[ 0.59089667 0.5340538 0. 0.7159462 0.712115
0.44310758 0.95455164]
[ 0.8068924 0.53641033 0. 1.1302563 0.69925725
0.7343639 0.40221703]
[ 0.93230265 0.84778297 0. 0.48484248 1.0201433
0.7651575 -0.00476191]
[ 0.74638265 0.33809522 0. -0.07576995 0.7840538
0.40910327 0.9659462 ]
[ 0.04544836 0.6931076 0. 1.0568924 -0.13025634
0.80074275 0.46358964]
[ 0.94925725 0.06769729 0. 0.26563603 1.0977829
-0.02014329 0.7348425 ]
[ 0.3534766 1.0151576 0. 0.74638265 0.67142856
1.0036174 0.25756338]
[ 0.7840538 0.7424366 0. 0.3787817 0.6931076
0.6212183 1.0568924 ]
[ 0.203077 0.80074275 0. 0.94925725 0.40103063
0.65221703 0.5989694 ]
[ 1.0977829 0.31319004 0. 0.68680996 1.0151576
0.66190475 0.74638265]
[ 1.0047619 1.0036174 0. 0.7840538 1.0757699
0.9659462 0.712115 ]
[ 0.6931076 0.95455164 0. 0.53641033 0.80074275
1.1302563 0.94925725]
[ 0.7343639 0.65221703 0. 1.0977829 0.64652336
0.7348425 1.0201433 ]
[ 1.0151576 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.8479438 0.67317617
0.5581815 0.1778345 ]
[-0.9215721 1.5896183 0. 0.6099795 0.5955366
-0.46569395 -0.8461083 ]
[ 1.6129647 1.4244858 0. 0.5209342 0.17585325
-0.8687666 1.7872683 ]
[ 1.3389692 0.8533131 0. -0.00590521 -0.7195761
1.6236191 1.1828533 ]
[ 1.1838211 0.6728102 0. -0.785988 1.2751837
1.1616383 0.933811 ]
[ 0.4684658 0.2719049 0. 1.2093123 0.66612804
0.66964823 0.55971766]
[ 0.17104894 -1.0688283 0. 0.6494252 0.6844874
0.66586125 0.01329695]
[-1.2607187 -0.22749203 0. -0.8741171 -0.9443728
-0.9659323 -0.03422031]
[-0.0364061 0.54829746 0. 0.6263525 0.66758543
0.04167109 -0.11780822]
[ 0.48400337 0.4685324 0. -0.04594427 0.02469592
-0.3487326 0.08831279]
[ 0.4161314 0.23332608 0. -0.13553022 -0.31008872
0.04969648 0.5674252 ]
[ 0.36492363 -0.07475745 0. -0.03859219 0.2016789
-0.39845943 -0.07058203]
[-0.08173721 0.1720942 0. 0.02323131 0.07122216
0.07469177 0.12792486]
[-0.24689877 0.196296 0. 0.5564647 0.535513
0.22528338 -0.37152448]
[-1.7235181 -1.8204601 0. -1.5040898 -1.8099409
-1.8550183 -1.1855855 ]
[-1.6341007 -1.3448519 0. -1.6656716 -1.6564709
-1.2735447 -1.3357594 ]
[-1.2829769 -1.2869868 0. -1.6657944 -1.4066424
-1.4230443 -1.4196167 ]
[-1.3691044 -1.656098 0. -1.4339573 -1.5685135
-1.633306 -1.4437945 ]]]]
推荐答案
错误是由于传递给函数readNetFromTensorflow的错误输入.pbtxt文件引起的，因为.pbtxt必须由 tf_text_graph_ssd.py 如描述
The error was caused by the wrong input .pbtxt file passed into the function readNetFromTensorflow because the .pbtxt has to be geneated by tf_text_graph_ssd.py as describe here:
Run this script to get a text graph of SSD model from TensorFlow Object Detection API. Then pass it with .pb file to cv::dnn::readNetFromTensorflow function.
对于其他模型，例如更快的r-cnn 和掩码r-cnn ，还有相应的脚本.
For other models such as faster r-cnn and mask r-cnn, there are also corresponding scripts.
PS:我刚刚发现有一个非常好的官方教程这里
PS: I just found there is a very good official tutorial here.
这篇关于重新训练MobileNet SSD V1 COCO后，Tensorflow的pb和pbtxt文件无法与OpenCV一起使用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
