How to export CV Mat to 4D-npy/npz, hdf5 or zarr?

1

For debugging purposes, I would like to have a 4D-CV Mat completely exported to an external file. I was thinking on numpy (npy/npz), hdf5 or zarr. Is that possible? Does someone have a code snipped which one can share?

(Background: the model.forward step of cv2.dnn results in a 4D CV Mat, where it is unclear to me how to slice the array)

2

if you are talking about python, net.forward() will result in a numpy array, you can simply use numpy.save()to serialize it
if it’s really c++, take a look at the tests, there’s npy code .
you could also use cv::FileStorage

tell us, what your network does. does it use some “known” architecture ?

3

Hello berak, thank you for the answer. I’ll try to explain my purpose and my approach.

The network does semantic segmentation with different output layers. Using the onnx-format I got (1,256,256,3) as output dimension (verified with python). As far as I understand the documentation it seems that NCHW is not supported at onnx as output layer.

For my working variant in C# (GitHub - shimat/opencvsharp: OpenCV wrapper for .NET) with protobuf,
I use the following with CvDnn.ReadNetFromTensorflow (output shape: 1, 3, 256, 256)
Mat[] layer = new Mat[2];
layer[0] = new Mat(256, 256, MatType.CV_32FC1, rawOutput.Ptr(0, 1));
layer[1] = new Mat(256, 256, MatType.CV_32FC1, rawOutput.Ptr(0, 2));

Now I want to switch to the onnx-format and access the different output layer.
Trying to convert the final CV Mat from hwc to chw and use the slicing above was not successful.

(adapted from opencv - Convert NCHW to NHWC in C++ - Stack Overflow)
public Mat hwc_to_chw(Mat src)
{
Mat dst = new Mat();
Mat[] channels = Cv2.Split(src);

// Stretch one-channel images to vector
foreach (var img in channels)
{
	img.Reshape(1, 1);
}

//// Concatenate three vectors to one
Cv2.HConcat(channels, dst);

return dst;

}​ ​

4

assuming, [1,3,256,256] means 3 color channels, i think, you wanted this instead (the opposite of that hwc_to_chw function :

Mat[] layer = new Mat[3];
layer[0] = new Mat(256, 256, MatType.CV_32FC1, rawOutput.Ptr(0, 0));
layer[1] = new Mat(256, 256, MatType.CV_32FC1, rawOutput.Ptr(0, 1));
layer[2] = new Mat(256, 256, MatType.CV_32FC1, rawOutput.Ptr(0, 2));

Mat res = new Mat;
Cv2.merge(layer, res); // res has "interleaved" channels

apart from that, a few questions:

  • “Using the onnx-format I got …” <–> “Now I want to switch to the onnx-format” – what, now ?
  • “As far as I understand the documentation it seems that NCHW is not supported at onnx as output layer.” – have a link ?
  • can you point us to some code / repo you’re trying to use ?
  • segmentation networks usually return masks (either a single channel labels mask, or a list of n per-class maps), – not color channels, what about it ?
5

Thank you very much for the answer. Probably my answer was a bit to unclear,
because I don’t think/understand cv2.merge might work.

For the NCHW-case ([1,3,256,256]) I’ve a C#-solution to get out
the correct slices of interest. In my case the first slice is only background.
Therefore I access slice 2 and 3 with
layer[1] = new Mat(256, 256, MatType.CV_32FC1, rawOutput.Ptr(0, 1));
layer[2] = new Mat(256, 256, MatType.CV_32FC1, rawOutput.Ptr(0, 2))

The output with Onnx is in the NWHC ([1,256,256,3]). Therefore my idea
was to change the cv mat to NCWH-case and then adresse again with
layer[1] = new Mat(256, 256, MatType.CV_32FC1, chwConverted.Ptr(0, 1));
layer[2] = new Mat(256, 256, MatType.CV_32FC1, chwConverted.Ptr(0, 2));

1 Like