Error when loading ONNX model with Upsample2D

,
1

Hello dear Team,

I have a problem with loading simple network, including Upsample2D. OpenCV DNN gives error:
dnn::readNet_12() : OpenCV(4.6.0-dev) C:\Users\satoo\Desktop\opencv\modules\dnn\src\onnx\onnx_importer.cpp:1040: error: (-2:Unspecified error) in function ‘cv::dnn::dnn4_v20220524::ONNXImporter::handleNode’

Node [Upsample@ai.onnx]:(onnx_node!Upsample__81) parse error: OpenCV(4.6.0-dev) C:\Users\satoo\Desktop\opencv\modules\dnn\src\layers\resize_layer.cpp:48: error: (-2:Unspecified error) in function ‘__cdecl cv::dnn::ResizeLayerImpl::ResizeLayerImpl(const class cv::dnn::dnn4_v20220524::LayerParams &)’

:
‘interpolation == “nearest” || interpolation == “opencv_linear” || interpolation == “bilinear”’
where
‘interpolation’ is linear

Please let me know workaround of this issue. I tried to generate ONNX from keras H5 model with different opsets, including 8,11,14,16. No any impact on error. But how I can see, Opencv DNN supports Upsampling2D operation. So please help me with conversion.

Regards,
Tigran

2

looks like you could try patching the network’s definition to say opencv_linear or bilinear instead of linear. or submit an issue on opencv’s github. disclaimer: I have no clue.

3

Thank you for your response, in my network I have only 3 UpSample2D, and in all of them I set interpolation=“bilinear”.

4

I opened case in OpenCVForUnity, they read ONNX also with opencv-python and got the same error, so they advise to get help here in forum.

5

yeah, you’re closer to a solution here than in any Unity forum.

still closer you get because you opened the issue :slight_smile:

though… are you sure you are using OpenCV v4.2? the error message says 4.6.0

by the way, “Upsample” was deprecated by ONNX. Why upsample operator was deprecated?? · Issue #2501 · onnx/onnx · GitHub

6

it gets a bit confusing now.
what did you do, exactly ?
(did you really change the keras model, re-train & re-export it ?)
and why does the error (still) say “linear” ?

7

i made some experiment:
(tl:dr; onnx is guilty)

#
# code from here: https://github.com/onnx/onnx/issues/2524
#
import torch
import torch.nn as nn
import torch.nn.functional as F
import os
class TestModel(nn.Module):
    def __init__(self):
        super(TestModel, self).__init__()
    def forward(self, x):
        x = F.interpolate(x, (256, 256), mode = 'bilinear')
        return x
torch_model = TestModel()
dummy_input = torch.randn((1, 3, 256, 256))
torch_out = torch.onnx.export(torch_model,
                              dummy_input,
                              'test_model.onnx',
                              verbose=True,
                              opset_version=11,)

output, look at the end of %11, watson:


graph(%x : Float(1, 3, 256, 256, strides=[196608, 65536, 256, 1], requires_grad=0, device=cpu),
      %12 : Long(2, strides=[1], requires_grad=0, device=cpu)):
  %2 : Long(4, strides=[1], device=cpu) = onnx::Shape(%x) # /usr/local/lib/python3.7/dist-packages/torch/nn/functional.py:3919:0
  %3 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={0}]() #
...
  %11 : Float(*, *, *, *, strides=[196608, 65536, 256, 1], requires_grad=0, device=cpu) = onnx::Resize[coordinate_transformation_mode="pytorch_half_pixel", cubic_coeff_a=-0.75, mode="linear", nearest_mode="floor"](%x, %9, %10, %8) # /usr/local/lib/python3.7/dist-packages/torch/nn/functional.py:3919:0
  return (%11)

so, the onnx exporter already switched bilinear to linear

next: solve the mystery, why the dnn’s onnx importer only supports ‘opencv_linear’, not a plain, obvious ‘linear’.

8

I recheck my CNN code and print layers in my keras model, so clearly all three Upsamples used in CNN, have directly mentioned parameter “bilinear”. But if you tested, that ONNX changes bilinear to linear, then it follows issue in ONNX. Thanks very much, I am going to try to solve issue in ONNX, do you have any ideas what to do for correcting ONNX change?

9

not at all, sorry.
i think, we’ll have to raise an issue with onnx, and let the devs there decide, what to do about it.

however, “linear” should be translated to “opencv_linear” in the dnn onnx importer:

it just does not happen, if you come from keras (only ptorch supported), i guess.
and this is the part, we can talk about with the opencv devs (e.g. in our issue !)

10

Hello guys,

In OpenCV github developers open and fix bug related this issue.
Please look through: