"Failed to initialize Inference Engine backend" on MyriadX

Python
1

Note: sorry for string links - since I created this account recently, I can’t include more than 2 links in one post.

Note 2: this question was originally asked on OpenVINO github.

Using OpenVINO 2022.2 with OpenCV 4.7.0 (both built from source) and Python 3.9.16 on Ubuntu 22.04.

I have a Movidius MyriadX VPU and an IP camera and want to run the inference on a live stream using MyriadX for object detection. I have downloaded and converted the model ssd_mobilenet_v1_coco (docs[dot]openvino[dot]ai/2022.2/omz_models_model_ssd_mobilenet_v1_coco.html) with

omz_downloader --name ssd_mobilenet_v1_coco --precisions FP16 --output_dir /home/user0/Downloads/ssd_mobilenet_v1_coco_downloaded

and

omz_converter --download_dir /home/user0/Downloads/ssd_mobilenet_v1_coco_downloaded --output_dir /home/user0/Downloads/ssd_mobilenet_v1_coco_converted --name ssd_mobilenet_v1_coco --precisions FP16

Then trying to run code attached in openvino_error.zip (zip includes model files too; in code some info is redacted, like the IP camera stream URL). It includes the following relevant lines:

net = cv2.dnn.readNetFromModelOptimizer(bin = model_bin, xml = config_xml)
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD)
...
video_stream = cv2.VideoCapture(filename = stream_url)
...
ret, frame = video_stream.read()
...
frame = cv2.resize(frame, (300,300), interpolation = cv2.INTER_AREA)
params = cv2.dnn.Image2BlobParams(scalefactor = 1.0/127.5, size = (300, 300), mean = 127.5, swapRB = True, ddepth = cv2.CV_32F, datalayout = cv2.dnn.DNN_LAYOUT_NHWC) 
blob = cv2.dnn.blobFromImageWithParams(frame, params)
net.setInput(blob)
detections = net.forward() # ERRORS HERE
...

Calling the program as

/home/user0/build-opencv/setup_vars.sh python3 opencv_app.py

but facing the error:

...
[ERROR]: OpenCV(4.7.0-dev) /home/user0/opencv/modules/dnn/src/ie_ngraph.cpp:865: error: (-2:Unspecified error) in function 'initPlugin'
> Failed to initialize Inference Engine backend (device = MYRIAD): Convert_5316 of type Convert: [ GENERAL_ERROR ]
> /home/jenkins/agent/workspace/private-ci/ie/build-linux-ubuntu20/b/repos/openvino/src/plugins/intel_myriad/graph_transformer/src/stages/convert.cpp:67 [Internal Error]: Final check for stage Convert_5316 with type Convert has failed: Conversion from FP16 to U8 is unsupported

As I understand, VPU only supports FP16 precision (intel[dot]com/content/www/us/en/developer/articles/technical/should-i-choose-fp16-or-fp32-for-my-deep-learning-model.html) hence when converting I specified that precision and am not sure what exactly is trying to convert FP16 to U8 and why it is being done.

I have converted the same model to FP32 (just switched --precisions FP32 in the omz_converter command) but the error persists. Using a different model - ssdlite_mobilenet_v2 (docs[dot]openvino[dot]ai/2022.2/omz_models_model_ssdlite_mobilenet_v2.html) - in FP16 did not solve the error either.

Any pointers appreciated!

2

Have you solved your problem?

What’s your platform?

3

No, haven’t solved it yet.

In terms of platform, output of uname -a is Linux XXXX 5.15.0-75-generic #82-Ubuntu SMP Tue Jun 6 23:10:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux. Versions are:

  • Ubuntu 22.04
  • Python 3.9.16
  • OpenCV 4.7.0 (built from source)
  • OpenVINO 2022.2 (built from source)

Let me know if more details are needed.

Myriad X VPU is being recognized by both lsusb and as one of the available_devices in openvino.runtime.Core().

from openvino import runtime
ie = runtime.Core()
for device in ie.available_devices:
    device_name = ie.get_property(device, "FULL_DEVICE_NAME")
    print(f"{device}: {device_name}")

output is

CPU:    Intel Atom(R) x6425RE Processor @ 1.90GHz
GNA: GNA_SW
MYRIAD: Intel Movidius Myriad X VPU

Additionally, the error is sometimes referring to Convert_5316, and sometimes to Convert_1132.

4

I have got an oak-d kit and error with your code is
[

INFO] ======== On CPU:
[INFO] accessing video stream...
[INFO] stream url:
[INFO] ======== On VPU:
[INFO] accessing video stream...
[INFO] stream url:
[ERROR]: OpenCV(4.8.0-pre) C:\lib\opencv\modules\dnn\src\ie_ngraph.cpp:865: error: (-2:Unspecified error) in function 'cv::dnn::InfEngineNgraphNet::initPlugin'
> Failed to initialize Inference Engine backend (device = MYRIAD): Convert_6159 of type Convert: [ GENERAL_ERROR ]
> C:\lib\openvino\src\plugins\intel_myriad\graph_transformer\src\stages\convert.cpp:70 [Internal Error]: Final check for stage Convert_6159 with type Convert has failed: Conversion from FP16 to U8 is unsupported
5

Thanks for confirming/reproducing!

If the issue is my code, do you know what in my code should be changed to avoid/resolve this error?

I will try to use a different approach and create a model using the openvino.runtime.Core()'s read_model function instead of cv2.dnn’s readNetFromModelOptimizer next.

6

The new approach I described in my earlier reply (using this tutorial as a reference) results in no errors, program runs successfully. I will need to verify if it in fact is running on Myriad X VPU.

New code here: opencv_error_20230628.zip

This is solved for now, but I still would like to know what causes the error with cv2.dnn.readNetFromModelOptimizer for future reference.

1 Like