Inferencing ONNX model on a RGB image in Android-Java

We have 3 channel input image of size 128x128 on which we want to run object detection Model (in ONNX format).

On printing inputImg Mat in Java, we get :
// Mat [ 128 x 128 x CV_8UC3, isCont=true, isSubmat=false, nativeObj=0x76e1a34500, dataAddr=0x76e28ba400 ]

Then we convert the intensity values in the range (0-1) :
inputImg.convertTo(inputImg, CvType.CV_32F, 1.0 / 255.0);

On printing inputImg in Java, we get :
// Mat [ 128 x 128 x CV_32FC3, isCont=true, isSubmat=false, nativeObj=0x76e1a34500, dataAddr=0x76e226ca00 ]

Now, we want to convert this image in blob for inferencing through object detection algorithm using OpenCV DNN module

Mat blob = Dnn.blobFromImage(inputImg, 1,
new org.opencv.core.Size(128, 128),
new Scalar(0, 0, 0), false, false, CV_32F);

// Mat [ 1 x 3 x 128 x 128 x CV_32FC1, isCont=true, isSubmat=false, nativeObj=0x76eca921c0, dataAddr=0x764bab8840 ]

Now we reshape this blob mat because our model expected 3 channels as Input instead of 128 (i.e 1x128x128x3)

Mat reshapeBlob;
int[ ] new_blobShape = {1, 128, 128, 3};
reshapeBlob = blob.reshape(1, new_blobShape);
// Mat [ 1 x 128 x 128 x 3 x CV_32FC1, isCont=true, isSubmat=false, nativeObj=0x7649b27d20, dataAddr=0x764a5c2980 ]

Then we set the reshapedBlob as input to the ONNX model and do forward pass :
Mat detections = net.forward();

We have tested this blob as input (in Python) to the same model and it works (i.e. the detection mat output as 1 x 896 x 140), but in Java, this blob is not giving the expected result it provides (1 x 896).

My question is if this is the correct way to reshape the blob and does
Mat [ 1 x 128 x 128 x 3 x CV_32FC1 ] represents blob of shape (1,128,128,3) ?

Kindly help with this issue.

your reshape is wrong, in any case !

what kind of object-detection network is it, exactly ?

  • if it wants NCHW input (like YOLO), your 4d blob is already in the correct shape
  • if it wants NHWC input (e.g. something from tf), you can’t simply reshape() it (memory needs to be reordered)
    use blobFromImageWithParams() (lets you choose NHWC order) or simply wrap a 4 dim Mat around your image data


most detection networks have several outputs (region proposals from different output layers), which you have to collect before evaluating, you only catch the last (most coarse) output layer

My model requires the NHWC order, Can you help me with how can I choose the NHWC order using blobFromImageWithParams() in Java?

have a look here

There is no method written to set the datalayout to DNN_LAYOUT_NHWC in JAVA I’m using Opencv 4.8 in android by default it return the NCHW.

How can I achieve NHWC from NCHW?

sadly, you’re right about it. not good. bug or omission.

for now assuming a single image, maybe you can construct the blob like:

Mat img_f; img.convertTo(img_f, CvType.CV_32F);
// apply mean / scale
int new_blobShape = {1, 128, 128, 3};
Mat blob = new Mat(blobShape, CVType.CV_32F, img_f.get(0,0));
sorry, could not find a smooth way to do this in java ,

Thank you for the answers.

  • Can you help me figure out in this Mat what CV_32FC1 signifies, my image has 3 channels but after making it a blob its shape is ( 1 x 3 x 128 x 128 x CV_32FC1).
  • Is it a 4D mat or a 5D mat?
  • Why does TransposingND() NCHW mat to NHWC mat not work on JAVA while in C++ /Python is working?
  • Can you please elaborate on what you meant by reordering the memory for reshape()?

N x C x H x W

CV_<type>C1 == CV_<type>