3D Image segmentation with 3D U-NET

C++
1

Hi everyone,

I am new with the DNN module of OpenCV and I am seeking to load a 3D U-NET model previously trained with tensorflow (python) to segment 3D medical images.
For now, I have successfully exported the model frozen graph and read it thanks to cv::dnn::readNetFromTensorflow. But, I still struggle to get a prediction.
My model takes in input 64x64x64 grayscale images with float pixels. As I haven’t seen any equivalent of cv::blobFromImage for 3D mat, I have tried to write my own blob conversion function from 64x64x64 mat like this:

cv::Mat blob;
  int sz[] = { 1, 1, patch.size[0],patch.size[1],patch.size[2] };
  blob.create( 5, sz, CV_32F );
  for(int i = 0; i < patch.size[0]; ++i) {
    for(int j = 0; j < patch.size[1]; ++j) {
      for(int k = 0; k < patch.size[2]; ++k) {
        int patchIdx[3] = { i,j,k };
        int blobIdx[5] = { 0, 0, i,j,k };
        blob.at<float>( blobIdx ) = patch.at<float>( patchIdx );
      }
    }
  }
  std::cerr << "Blob: " << blob.size << std::endl; // return Blob: 1 x 1 x 64 x 64 x 64

Unfortunately, when I call:
net.setInput( blob);
cv::Mat result = net.forward();
I get this error:

[ERROR:0] global C:\Libs-VS17\opencv\opencv\modules\dnn\src\dnn.cpp (3512) cv::dnn::dnn4_v20210301::Net::Impl::getLayerShapesRecursively OPENCV/DNN: [Convolution]:(model/conv3d_8/Conv3D): getMemoryShapes() throws exception. inputs=1 outputs=0/1 blobs=2
[ERROR:0] global C:\Libs-VS17\opencv\opencv\modules\dnn\src\dnn.cpp (3515) cv::dnn::dnn4_v20210301::Net::Impl::getLayerShapesRecursively input[0] = [ 1 32 16 16 128 ]
[ERROR:0] global C:\Libs-VS17\opencv\opencv\modules\dnn\src\dnn.cpp (3523) cv::dnn::dnn4_v20210301::Net::Impl::getLayerShapesRecursively blobs[0] = CV_32FC1 [ 128 256 2 2 2 ]
[ERROR:0] global C:\Libs-VS17\opencv\opencv\modules\dnn\src\dnn.cpp (3523) cv::dnn::dnn4_v20210301::Net::Impl::getLayerShapesRecursively blobs[1] = CV_32FC1 [ 128 1 ]
[ERROR:0] global C:\Libs-VS17\opencv\opencv\modules\dnn\src\dnn.cpp (3525) cv::dnn::dnn4_v20210301::Net::Impl::getLayerShapesRecursively Exception message: OpenCV(4.5.2) C:\Libs-VS17\opencv\opencv\modules\dnn\src\layers\convolution_layer.cpp:386: error: (-2:Unspecified error) Number of input channels should be multiple of 256 but got 32 in function ‘cv::dnn::ConvolutionLayerImpl::getMemoryShapes’
OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(4.5.2) Error: Unspecified error (Number of input channels should be multiple of 256 but got 32) in cv::dnn::ConvolutionLayerImpl::getMemoryShapes, file C:\Libs-VS17\opencv\opencv\modules\dnn\src\layers\convolution_layer.cpp, line 386

As the error occured at the con3d_8 layer it seems the given blob format is correct but something get wrong during the layers sequence.
Has anyone an idea of what is wrong in this implementation ?
Thx for your time.