Hi all,
System information (version)
*TensorFlow => 2.4.1
- OpenCV => 4.5.1
- Operating System / Platform => Windows 10 64 Bit
- Compiler => Visual Studio 2019
Detailed description
For my task I want to use the Mask R-CNN Inception ResNet V2 1024x1024 object detection model from the TensorFlow 2 Detection Zoo.
I already set up everything regarding TensorFlow by following this tutorial: Train a Mask R-CNN model with the Tensorflow Object Detection API
I can also use TensorFlow to run the inference, e.g. for a single image. So everything works on the TensorFlow side.
Now I want to port my SavedModel from TensorFlow to C++ and OpenCV DNN. And I’ve been failing at this for several days now, I’ve checked and tried quite a few forum posts and Github issues, but nothing could solve my problems.
But let’s start from the beginning:
I know that the OpenCV DNN module does not support the SavedModel format and does expect the FrozenGraph format instead. I have already some problems with the conversion from SavedModel to FrozenGraph:
To convert the SavedModel to a FrozenGraph I use the following python code:
import tensorflow as tf
from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2
loaded = tf.saved_model.load('inference_graph/saved_model')
infer = loaded.signatures['serving_default']
f = tf.function(infer).get_concrete_function(input_tensor=tf.TensorSpec(shape=[1, None, None, 3], dtype=tf.uint8))
f2 = convert_variables_to_constants_v2(f)
graph_def = f2.graph.as_graph_def()
# Export frozen graph
with tf.io.gfile.GFile('frozen_graph.pb', 'wb') as f:
f.write(graph_def.SerializeToString())
I found this code as an answer to a stackoverflow issue.
‘inference_graph/saved_model’ is the path to the folder containing my SavedModel. This code saves the resulting FrozenGraph as frozen_graph.pb. Everything seems to work fine.
BUT when I run the following code with my newly gained FrozenGraph:
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/dnn.hpp>
#include <iostream>
using namespace cv;
using namespace cv::dnn;
using namespace std;
int main()
{
// Path to the FrozenGraph file
String savedModelPath= ".\\pretrained_TF-model\\frozen_graph.pb";
Net net = readNetFromTensorflow(savedModelPath);
net.setPreferableBackend(DNN_BACKEND_OPENCV);
net.setPreferableTarget(DNN_TARGET_CPU);
Mat img = imread(".\\images\\Screenshot.png");
Mat blobImg = blobFromImage(img, 1.0, Size(512, 512), Scalar(0, 0, 0),false,false, CV_32F);
net.setInput(blobImg);
Mat output = net.forward();
return 0;
}
The readNetFromTensorflow() method throws an exception difficult to decipher for me:
OpenCV(4.5.1) Error: Unspecified error (Input layer not found: StatefulPartitionedCall/StatefulPartitionedCall/Const_1) in cv::dnn::dnn4_v20201117::`anonymous-namespace’::TFImporter::connect, file C:\build\master_winpack-build-win64-vc15\opencv\modules\dnn\src\tensorflow\tf_importer.cpp, line 553
However, there is also the possibility to give this method a second parameter, the so-called config file.
This config file needs to be generated using the FrozenGraph file. However, I do not manage to generate this file. If I use the “tf_text_graph_mask_rcnn.py” script as described [here] (TensorFlow Object Detection API · opencv/opencv Wiki · GitHub), I get the
KeyError: ‘first_stage_features_stride’ in line 41
no matter which of the two config files I specify: pipeline.config or mask_rcnn_inception_resnet_v2_1024x1024_coco17_gpu-8.config. I don’t know which of these two files is the correct one, different sources refer to different files. But these are the only *.config files I have.
Summary
No matter what I googled and tried, I kept running into problems. So I still don’t know where the error lies: with the conversion, or with my C++/OpenCV code. However, I strongly suspect that it is the conversion.
Can anyone help me?
Has anyone had similar problems?
Should I open an issue, because all the issues I have seen do not deal with this specific problem regarding the Mask R-CNN model? And if so, what kind of issue should I open, is it because of the conversion, or is the error with the readNetFromTensorflow method?
Thank you in advance,
Jonas