# After processing the semantic segmentation results of Deeplabv3 using Argmax, the results were incorrect. Please help me take a look

**URL:** <https://forum.opencv.org/t/after-processing-the-semantic-segmentation-results-of-deeplabv3-using-argmax-the-results-were-incorrect-please-help-me-take-a-look/13136>\
**Category:** C++\
**Tags:** dnn, segmentation, core\
**Created:** [May 11, 2023, 3:36am UTC](https://forum.opencv.org/t/after-processing-the-semantic-segmentation-results-of-deeplabv3-using-argmax-the-results-were-incorrect-please-help-me-take-a-look/13136 "2023-05-11T03:36:50Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ZJD](https://avatars.discourse-cdn.com/v4/letter/z/87869e/32.png) [@ZJD](https://forum.opencv.org/u/ZJD)\
**Post date:** [May 11, 2023, 3:36am UTC](https://forum.opencv.org/t/after-processing-the-semantic-segmentation-results-of-deeplabv3-using-argmax-the-results-were-incorrect-please-help-me-take-a-look/13136/1 "2023-05-11T03:36:51Z")

</div>

The inference statement used is this：

```auto
Mat score = net.forward("outputs");

```

The output format of the model is float [1x2x512x648]  
The input image looks like this：

I can parse the correct results using these few codes.

```auto
cv::Mat output1(512, 648, CV_32F, (float*)score.data);
cv::Mat output2(512, 648, CV_32F, (float*)score.data+648*512);
cv::Mat sum = output2 - output1;
cv::threshold(sum, sum, 0, 255, cv::THRESH_BINARY);

```

But using the method I wrote myself, the parsing result is incorrect. May I know how to modify my code.

```auto
const int OUTPUT_H = 512;
const int OUTPUT_W = 648;
const int NUM_CLASSES = 2;

void postprocess(const uchar* output, Mat& result)
{
	result.create(OUTPUT_H, OUTPUT_W, CV_8U);
	//result.create(OUTPUT_H, OUTPUT_W, CV_32SC1);
	for (int i = 0; i < OUTPUT_H; ++i) {
		for (int j = 0; j < OUTPUT_W; ++j) {
			int idx = i * OUTPUT_W + j;
			int max_idx = -1;
			float max_val = -FLT_MAX;
			for (int k = 0; k < NUM_CLASSES; ++k) {
				float val = output[idx * NUM_CLASSES + k];
				if (val > max_val) {
					max_val = val;
					max_idx = k;
				}
			}
			result.at<uint8_t>(i, j) = max_idx * 100;
			//result.at<int>(i, j) = max_idx * 100;
		}
	}
}

```

```auto

// Postprocess output
Mat result;
postprocess(score.data, result);

```

please help me.  
The src image,result image, output1 image, output2 image ,sum image like this：

 ![image](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/0/07109e6e97a61f9e4e5e16e773d069b83bfc3b32.jpeg)

---

<div class="post-metadata">

**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\
**Post date:** [May 11, 2023, 5:47am UTC](https://forum.opencv.org/t/after-processing-the-semantic-segmentation-results-of-deeplabv3-using-argmax-the-results-were-incorrect-please-help-me-take-a-look/13136/2 "2023-05-11T05:47:24Z")

</div>

network output is float32, you treat it as it were uchar:

> [@ZJD](#):
>
> `void postprocess(const uchar* output,`

so wrong values (wrong type) here:

> [@ZJD](#):
>
> `float val = output[idx * NUM_CLASSES + k];`

also, this looks wrong. what did you want here ?

> [@ZJD](#):
>
> `idx * NUM_CLASSES + k`

do you have a link to the generator / export code, please ?

---

<div class="post-metadata">

**Author:** ![ZJD](https://avatars.discourse-cdn.com/v4/letter/z/87869e/32.png) [@ZJD](https://forum.opencv.org/u/ZJD)\
**Post date:** [May 11, 2023, 6:30am UTC](https://forum.opencv.org/t/after-processing-the-semantic-segmentation-results-of-deeplabv3-using-argmax-the-results-were-incorrect-please-help-me-take-a-look/13136/3 "2023-05-11T06:30:11Z")

</div>

I have modified the data type, but the result is still incorrect.  
`void postprocess(const float* output, Mat& result)`

```auto
// Postprocess output
Mat result;
const float* data = reinterpret_cast<const float*>(score.ptr());
postprocess(data, result);

```

 ![image](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/4/44fc730ff4f3c27d8461f72e854b04c76bed0bd6.png)

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [May 11, 2023, 6:39am UTC](https://forum.opencv.org/t/after-processing-the-semantic-segmentation-results-of-deeplabv3-using-argmax-the-results-were-incorrect-please-help-me-take-a-look/13136/4 "2023-05-11T06:39:40Z")

</div>

did you also change the `postprocess` function? please show the current code.

---

<div class="post-metadata">

**Author:** ![ZJD](https://avatars.discourse-cdn.com/v4/letter/z/87869e/32.png) [@ZJD](https://forum.opencv.org/u/ZJD)\
**Post date:** [May 11, 2023, 6:44am UTC](https://forum.opencv.org/t/after-processing-the-semantic-segmentation-results-of-deeplabv3-using-argmax-the-results-were-incorrect-please-help-me-take-a-look/13136/5 "2023-05-11T06:44:58Z")

</div>

```auto

void postprocess(const float* output, Mat& result)
{
	result.create(OUTPUT_H, OUTPUT_W, CV_8U);
	//result.create(OUTPUT_H, OUTPUT_W, CV_32SC1);
	for (int i = 0; i < OUTPUT_H; ++i) {
		for (int j = 0; j < OUTPUT_W; ++j) {
			int idx = i * OUTPUT_W + j;
			int max_idx = -1;
			float max_val = -FLT_MAX;
			for (int k = 0; k < NUM_CLASSES; ++k) {
				float val = output[idx * NUM_CLASSES + k];
				if (val > max_val) {
					max_val = val;
					max_idx = k;
				}
			}
			result.at<uint8_t>(i, j) = max_idx * 100;
			//result.at<int>(i, j) = max_idx * 100;
		}
	}
}

```

```auto

int main() {

……
Mat score = net.forward("outputs");

// Postprocess output
Mat result;
const float* data = reinterpret_cast<const float*>(score.ptr());
postprocess(data, result);

……
cv::Mat output1(512, 648, CV_32F, (float*)score.data);|
cv::Mat output2(512, 648, CV_32F, (float*)score.data+648*512);|
cv::Mat sum = output2 - output1;|
cv::threshold(sum, sum, 0, 255, cv::THRESH_BINARY);|

……
}

```

 ![image](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/7/769a34028615a95c2f673c1cd7e062d37f9d5797.png)

---

<div class="post-metadata">

**Author:** ![ZJD](https://avatars.discourse-cdn.com/v4/letter/z/87869e/32.png) [@ZJD](https://forum.opencv.org/u/ZJD)\
**Post date:** [May 11, 2023, 7:18am UTC](https://forum.opencv.org/t/after-processing-the-semantic-segmentation-results-of-deeplabv3-using-argmax-the-results-were-incorrect-please-help-me-take-a-look/13136/7 "2023-05-11T07:18:52Z")

</div>

The image composition in output1 and output2 has different probabilities, one probability being the background layer and the other probability being the desired result. The semantic segmentation of deeplab is to calculate the Argmax for different results. Taking the value with the highest probability as the result may require some AI knowledge to understand, and its result is in the form of a 4-dimensional matrix NCHW. Because I don’t know how to parse its results, the core code is to refer to others’ semantic segmentation and parsing, and I don’t quite understand either.

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [May 11, 2023, 11:46am UTC](https://forum.opencv.org/t/after-processing-the-semantic-segmentation-results-of-deeplabv3-using-argmax-the-results-were-incorrect-please-help-me-take-a-look/13136/8 "2023-05-11T11:46:09Z")

</div>

I’d recommend dumping that `float` array to a tiff (tiff can do floats and multiple channels/layers/pages) and looking at the data using some tool… python at least because that’s a lot less of a headache when it comes to dealing with arrays and numbers. you could also upload the data.

---

<div class="post-metadata">

**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\
**Post date:** [May 11, 2023, 12:34pm UTC](https://forum.opencv.org/t/after-processing-the-semantic-segmentation-results-of-deeplabv3-using-argmax-the-results-were-incorrect-please-help-me-take-a-look/13136/9 "2023-05-11T12:34:50Z")

</div>

> [@ZJD](#):
>
> `idx * NUM_CLASSES + k`

this still calculates _interleaved_ probs, like p1,p2,p1,p2,p1,p2,…  
while there is a whole WxH plane lying between pixels at the same pos between output1 & output2  
isn’t it rather:

```
idx + (k * W * H)

```

?