Model inference resulting in unknown rows and cols

C++
1

I converted a Tensorflow Model to ONNX and then load it using OpenCV DNN. This model expects a [-1, 100, 100, 1] input and gives a [-1, 100, 100 ,1] output

digs
digs1576×688 30.9 KB

I tested in Python, creating a dummy input and I got exactly what I expected, a 100x100 output.

...
params = cv2.dnn.Image2BlobParams()
params.mean = (0,0,0)
params.scalefactor = (1,1,1)
params.size = (100, 100)
params.ddepth = cv2.CV_32F
params.datalayout = cv2.dnn.DNN_LAYOUT_NHWC
params.paddingmode = cv2.dnn.DNN_PMODE_NULL
...
batch_size = 1
h = 100
w = 100
c = 1
x = torch.rand(batch_size, h, w, c, requires_grad=False)
x_npy = x.detach().numpy()
x_opencv = x_npy.reshape(h, w, 1)

blobPB = cv2.dnn.blobFromImageWithParams(x_opencv, params)
modelopenCV.setInput(blobPB)
output = modelopenCV.forward()
print(output.shape) # = (1, 100, 100, 1)

But I’m having issue in acessing my output elements in C++, my cv::Mat output from model.foward() gets -1 rows and -1 cols. Any idea of how can I acess this elements? Why this difference between Python and C++ result? I know that OpenCV Python uses np.arrays instead of cv::Mat for data storage, but I guess it should has the same dimensions of a cv::Mat, using a input with the correct dimensions and the same model… no?

...
		params.scalefactor = 1.0 / 255;
		params.size = cv::Size2i::Size_(100, 100);
		params.mean = 0.0;
		params.ddepth = CV_32F;
		params.datalayout = cv::dnn::DNN_LAYOUT_NHWC;
		params.paddingmode = cv::dnn::DNN_PMODE_NULL;
        cv::dnn::blobFromImageWithParams(imgpb, blobTest, params);
        /* trust me imgpb it's in correct dimensions, imgpb.rows = 100, imgpb.cols = 100, imgpb.channels() = 1 */
		pModelONNXPeB.setInput(blobTest);
	    cv::Mat outputsPB1 = pModelONNXPeB.forward();

outputsPB1 has 4 dimensions, -1 rows and -1 cols

2

opencv sets rows & cols to -1 for more-than-2-dim Mats.
please use outputsPB1.size (w/o braces !) in the same way you would use xxx.shape in python

3

Didn’t know that… thanks, Berak. Could you give me an example of the syntax for acessing an element in 4 dim struct? I’m used to

	for (int i = 0; i < outputsPB1.rows; ++i) {
					for (int j = 0; j < outputsPB1.cols; ++j) {
						float valor = outputsPB1.at<float>(i, j);

But as you said, -1 rols and -1 cols are setted for > 2 dims, I can’t use this in this case.

4

you can use at<float>(i,j,k) , iterate over outputsPB1.shape[0](batch dim) etc.

but in your case, you probably want to make a simple [100,100] 2d Mat from it:

Mat res2d(100, 100, CV_32F, outputsPB1.ptr<float>(0)); // batch item 0
1 Like
5

May be you can try this

/**
* return a specific Mat in a blob.
* if dims blob is less or equal  to 2 (N rows x M columns) blob is return
* if dims blob is 3 blob is blob is (H x N x M array) mat at (coord(0),0, 0) is returned.
* if dims blob is r blob is blob is (T x H x N x M array) mat at (coord(0), coord(1), 0, 0) is returned.
* */
Mat getMatInBlob(Mat blob, vector<int> coord, int posChannel = DNN_LAYOUT_NCHW)
{
 }
1 Like
6

here you should try

modelopenCV.setInput(cv.Mat(blobPB))

7

Really good! A generic implementation, I will save for later, thanks for the tip. I don’t know, if I can reutilize this topic for another question, but in this same context of 4d data structs, is there anyway to pass a 4d data to a blob? blobFromImages only accepts structs with dims <= 2, right? I got this 4 dimension output and I need to create a blob from it to pass as input for another model.

8

A blob is a Mat
So you can do :

  Mat maskInput(vector<int> {1, 1, 256, 256}, CV_32FC1);
  netMask.setInput(maskInput, "mask_input");