OpenCV and deep learning neural networks

Hi to everyone!!! I am pretty new to OpenCV and object detection algorithm., i was able to run some code examples with face detection and recognition using openCV and haarcascades classifiers, but i suppose this methods are “deprecated” and more stable algorithms are based on deep learning and neural networks. When i start to investigate this question i stucked with a huge variations and combinations of deep learning algorithms and i don’t really know from what point to start and what is the best choise for me…

My goal is to make a face mask detection software based on OpenCV and deep learning network model using C++ language (please don’t suggest python).

Could you please help me to find the best combination for my task? I need the whole pipeline: building and training my own deep learning neural network model from scratch (or get pretrained), run openCV detection with that model.

Thank you very much!!! I working on cross platform C++ project in windows PC using Visual Studio 2019.

well at least for that part you won’t be able to avoid coding in python
(as you need to use some framework like tf or pytorch for the (re-)training)
((but hey, it’s all C, if you stick with YOLO…))

try to find a simple tutorial on “object-detection” networks

1 Like

I already found some tutorials with different bunches ncnn+opencv, tensorflow+opencv, but all examples predominantly with python code.

Thank you for a tip about the python coding, i am ready to learn how to create and train model with python, but my main project needs to be in C++.

Could you please do me a favour and advice which bunch to choose for my task:

  1. ncnn+openCV
  2. tensorflow + openCV
  3. caffe + openCV
  4. dnn + openCV

And how to choose from a huge plenty of neural networks:

  1. R-CNN
  2. YOLO
  3. HOG
  4. R-FCN
  5. SSD

What is the most popular practice? Help to choose please

  1. i dont know it, but it looks like inference only (no training there)
  2. we have a “train your own face detection” sample here .
    but no good support for tf>1.5 in opencv
  3. caffe is dead
  4. as 1. no training possible
  5. PyTorch
  6. YOLO (again, if you prefer C …)

cmon mate, just ask the right questions

  • you probably dont need to train from scratch (“fine-tuning”)
  • maybe you can find a pre-trained model, so you can already work on the inference code
2 Likes

Thank you for a valuable tips and explanation

Finally i decided to start with the following bunch: OpenCV DNN module + Lobe.ai

I successfully trained the model with Lobe (https://www.lobe.ai/) and make two exports for a test one export is a tensorflow “model.pb” format, another is “model.onnx” format, but both of them doesn’t work, i stucked even on model loading part of the code.

Tensorflow load example:

    // load the neural network model
    auto model = cv::dnn::readNet("input/saved_model.pb");
    
    // also try this way
     auto model = cv::dnn::readNetFromTensorflow("input/saved_model.pb");

ONNX load example:

 // load the neural network model
    auto model = cv::dnn::readNetFromONNX("input/model.onnx");

Please help me to win the errors. Thank you

Here is the errors attached when i hit the readNet functions

ONNX errors:

[ERROR:0] global D:\build\opencv_next\modules\dnn\src\onnx\onnx_importer.cpp (2110) cv::dnn::dnn5_v20210608::ONNXImporter::handleNode DNN/ONNX: ERROR during processing node with 1 inputs and 1 outputs: [ReduceMean]:(saved_resnet_50_model/resnet50v2/avg_pool/Mean:0)
OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(5.0.0-pre) Error: Unspecified error (> Node [ReduceMean]:(saved_resnet_50_model/resnet50v2/avg_pool/Mean:0) parse error: OpenCV(5.0.0-pre) D:\build\opencv_next\modules\dnn\src\onnx\onnx_importer.cpp:725: error: (-215:Assertion failed) axis >= 2 + i in function 'cv::dnn::dnn5_v20210608::ONNXImporter::handleNode'
> ) in cv::dnn::dnn5_v20210608::ONNXImporter::handleNode, file D:\build\opencv_next\modules\dnn\src\onnx\onnx_importer.cpp, line 2129

Tensorflow errors:

[libprotobuf ERROR D:\build\opencv_next\3rdparty\protobuf\src\google\protobuf\wire_format_lite.cc:629] String field 'opencv_tensorflow.FunctionDef.Node.ret' contains invalid UTF-8 data when parsing a protocol buffer. Use the 'bytes' type if you intend to send raw bytes.
OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(5.0.0-pre) Error: Unspecified error (FAILED: ReadProtoFromBinaryFile(param_file, param). Failed to parse GraphDef file: input2/saved_model.pb) in cv::dnn::ReadTFNetParamsFromBinaryFileOrDie, file D:\build\opencv_next\modules\dnn\src\tensorflow\tf_io.cpp, line 42

can you replace the sceenshots (hated ANYWHERE !) with a TEXT version, so folks here can look up your errors (impossible, given an image ) ? thank you.

1 Like

Yes, sure! Thank you

1 Like

unfortunately, its unclear, what you did there, or even what they are using.
impossible to debug
as of now, there is also classification only, right ?

importing tf2 saved_model is not fully supported yet

in the end, rather use something tried and tested

1 Like