# Action recognize with NN (2plus1d\_34)

**URL:** <https://forum.opencv.org/t/action-recognize-with-nn-2plus1d-34/12956>\
**Category:** C++\
**Tags:** dnn\
**Created:** [April 26, 2023, 7:37am UTC](https://forum.opencv.org/t/action-recognize-with-nn-2plus1d-34/12956 "2023-04-26T07:37:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ilya71](https://avatars.discourse-cdn.com/v4/letter/i/9e8a1a/32.png) [@ilya71](https://forum.opencv.org/u/ilya71)\
**Post date:** [April 26, 2023, 7:37am UTC](https://forum.opencv.org/t/action-recognize-with-nn-2plus1d-34/12956/1 "2023-04-26T07:37:25Z")

</div>

I have retrained the 2plus1d\_34 network using this notebook ([computervision-recipes/01\_training\_introduction.ipynb at staging · microsoft/computervision-recipes · GitHub](https://github.com/microsoft/computervision-recipes/blob/staging/scenarios/action_recognition/01_training_introduction.ipynb)). Saved the model in ONNX format:

```auto
    def save(self, model_path: Union[Path, str]) -> None:
        """ Save the model to a path on disk. """
        #torch.save(self.model.state_dict(), model_path)
        dummy_input = Variable(torch.randn(1, 3, 8, 112, 112, device="cuda"))
        torch.onnx.export(self.model, dummy_input, model_path)

```

I run this model using opencv c++ and python onnx and have two different results. Could you tell me what I’m doing wrong. Also, I did not work with arrays of dimension larger than the usual image, and here it was necessary to create a 5d array. I don’t really understand why sizes[5] and dim = 4.

c++:

```auto
auto net = cv::dnn::readNetFromONNX("test_model.onnx");
int sizes[5] = {1, 3, 8, 112, 112 };
cv::Mat w_i = cv::Mat(4, sizes, CV_32FC1, cv::Scalar(0));

for (int k = 0; k < 8; k++) {
  for (int i = 0; i < 112; i++) {
      for (int j = 0; j < 112; j++) {
  
          w_i.at<float>(cv::Vec<int, 5>(0, 0, k, i, j)) = 0.0;
          w_i.at<float>(cv::Vec<int, 5>(0, 1, k, i, j)) = 0.0;
          w_i.at<float>(cv::Vec<int, 5>(0, 2, k, i, j)) = 0.0;
      }
  }
}
net.setInput(w_i);

cv::Mat out;
net.forward(out);
std::cout << out << std::endl;

```

python:

```auto
import torch
import onnxruntime as ort

ort_sess = ort.InferenceSession('test_model.onnx')
frame = np.zeros((1,3,8,112,112))

input_data = frame
input_data = np.array(input_data, dtype=np.float32)

session = ort.InferenceSession('test_model.onnx')
output_data = session.run(None, {session.get_inputs()[0].name: input_data})

print(output_data)

```

if I create a matrix of the wrong size in c++ (for example 4D), then I will get this error:  
[ERROR:0] OPENCV/DNN: [Convolution]:(640): getMemoryShapes() throws exception. inputs=1 outputs=0/1 blobs=2  
[ERROR:0] input[0] = [3 8 112 112]  
[ERROR:0] blobs[0] = CV\_32FC1 [45 3 1 7 7]  
[ERROR:0] blobs[1] = CV\_32FC1 [45 1]  
[ERROR:0] Exception message: OpenCV(3.4.10) c:\tarhan\compiling\opencv\_3410\opencv-3.4.10\modules\dnn\src\layers\convolution\_layer.cpp:306: error: (-2:Unspecified error) Number of input channels should be multiple of 3 but got 8 in function ‘cv::dnn::ConvolutionLayerImpl::getMemoryShapes’

I don’t understand why there are two blobs here and why they don’t match what I saved

---

<div class="post-metadata">

**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\
**Post date:** [April 26, 2023, 8:33am UTC](https://forum.opencv.org/t/action-recognize-with-nn-2plus1d-34/12956/2 "2023-04-26T08:33:09Z")

</div>

a link to the exported onnx, and an example image sequence would be quite helpful here.

> [@ilya71](#):
>
> Also, I did not work with arrays of dimension larger than the usual image,

due to the onnx export, which ‘freezes’ the dimensions. you cant change image / time\_series size after that

> [@ilya71](#):
>
> I don’t really understand why sizes[5] and dim = 4.

that simply looks wrong, it must be  
`cv::Mat w_i = cv::Mat(5, sizes,`…

you’re writing out-of-bounds in the following for - loops

> [@ilya71](#):
>
> OpenCV(3.4.10)

oh, noes, quite outdated. please use recent 4.7 instead  
(also check / update the python cv2 version !)

---

<div class="post-metadata">

**Author:** ![ilya71](https://avatars.discourse-cdn.com/v4/letter/i/9e8a1a/32.png) [@ilya71](https://forum.opencv.org/u/ilya71)\
**Post date:** [April 26, 2023, 11:31am UTC](https://forum.opencv.org/t/action-recognize-with-nn-2plus1d-34/12956/3 "2023-04-26T11:31:03Z")

</div>

here is a link to my model: [test\_model.onnx - Google Drive](https://drive.google.com/file/d/1-2TSHFqeCJr6AI4t_BJ1jlPYya0wAHxK/view?usp=share_link)

I use an array filled with zeros as a sequence of images, because I just want to make sure that the python and c++ code produces the same result

> [@berak](#):
>
> that simply looks wrong, it must be  
> `cv::Mat w_i = cv::Mat(5, sizes,`…
> 
> you’re writing out-of-bounds in the following for - loops

when I do dims = 5 I get this error : std::exception: OpenCV(3.4.10) c:\tarhan\compiling\opencv\_3410\opencv-3.4.10\modules\dnn\src\dnn.cpp:2997: error: (-215:Assertion failed) total(os[i]) \> 0 in function ‘cv::dnn::experimental\_dnn\_34\_v17::Net::Impl::getLayerShapesRecursively’

> [@berak](#):
>
> oh, noes, quite outdated. please use recent 4.7 instead  
> (also check / update the python cv2 version !)

unfortunately, I can’t just raise the opencv version as this will lead to big changes in my project.