# Camera2 byteArray to BGR uint8 mat conversion

**URL:** <https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823>\
**Category:** Android/Java\
**Tags:** android\
**Created:** [August 12, 2022, 6:46am UTC](https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823 "2022-08-12T06:46:25Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Haider\_Saleem](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/haider_saleem/32/5847_2.png) [@Haider\_Saleem](https://forum.opencv.org/u/Haider_Saleem)\
**Post date:** [August 12, 2022, 6:46am UTC](https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823/1 "2022-08-12T06:46:25Z")

</div>

I’m trying to convert byteArray from camera2 onImageAvailable listener to Mat object, and then passing it to an algorithm for dehazing in c++. I have tried different methods available for converting byteArray to Mat channel 3 object but whenever I split the mat object in 3 channels then all of them get filled with garbage data which further cause crash.

Here’re the different methods used for converting byte array to mat

```
    val bmp = BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
    val orig = Mat(bmp.height, bmp.width, CvType.CV_8UC3)
    val myBitmap32 = bmp.copy(Bitmap.Config.ARGB_8888, true)
    Utils.bitmapToMat(myBitmap32,orig)

```

Using imread

```
        val matImage = Imgcodecs.imread(file!!.absolutePath,IMREAD_COLOR)

```

using decodebyte array

```
val bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
Utils.bitmapToMat(bitmap,matImage)

```

here’s c code for the JNICALL

```
dehazing(JNIEnv *env, jobject, jlong input, jlong output)
{
    cv::Mat& mInput = *((cv::Mat*) input);
    cv::Mat& mOutput = *((cv::Mat*) output);
    dehaze(mInput, mOutput);
}

```

and finally here’s a little chunk of c++ code

```
Mat dehazedSplit[3];
Mat ABGR[3];
Mat tBGR[3];
Mat imageBGR[3];
Mat blurredImageBGR[3];
// Normalize, blur image and extract dimensions
W = image.cols;
H = image.rows;

image.convertTo(image, CV_64FC3);
image = image / Scalar(255, 255, 255);

GaussianBlur(image, blurredImage, Size(41, 41), 30, 30);
split(blurredImage, blurredImageBGR);

// Estimate the A matrix
A = Mat(H, W, CV_64FC3, Scalar(1, 1, 1));

split(A, ABGR);

minMaxLoc(blurredImageBGR[0], &minVal, &maxVal, &minLoc, &maxLoc);
ABGR[0] = Scalar(maxVal);
minMaxLoc(blurredImageBGR[1], &minVal, &maxVal, &minLoc, &maxLoc);
ABGR[1] = Scalar(maxVal);
minMaxLoc(blurredImageBGR[2], &minVal, &maxVal, &minLoc, &maxLoc);
ABGR[2] = Scalar(maxVal);

// Estimate the t matrix
t = Mat(H, W, CV_64FC3, Scalar(0, 0, 0));
split(t, tBGR);

tBGR[0] = (Scalar(1) - blurredImageBGR[0].mul(Scalar(twBlue)) / ABGR[0]);
tBGR[1] = (Scalar(1) - blurredImageBGR[1].mul(Scalar(twGreen)) / ABGR[1]);
tBGR[2] = (Scalar(1) - blurredImageBGR[2].mul(Scalar(twRed)) / ABGR[2]);

```

Any suggestion/help will be appreciated.

---

<div class="post-metadata">

**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\
**Post date:** [August 12, 2022, 8:24am UTC](https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823/2 "2022-08-12T08:24:32Z")

</div>

i see various problems here:

> [@Haider\_Saleem](#):
>
> ```auto
> val orig = Mat(bmp.height, bmp.width, CvType.CV_8UC3)
> val myBitmap32 = bmp.copy(Bitmap.Config.ARGB_8888, true)
>  
> 
> ```

ARGB has 4 channels, it wont fit into CV\_8U3, so (i guess), it will ignore your allocation, and return a 4 channel Mat !

then, later:

> [@Haider\_Saleem](#):
>
> ```auto
> split(blurredImage, blurredImageBGR);
> 
> ```

again, input is ARGB, but blurredImageBGR has only 3 elems.  
undefined behaviour, nasal demons, segfaults, anything can happen

> [@Haider\_Saleem](#):
>
> ```auto
> minMaxLoc(blurredImageBGR[0], &minVal, &maxVal, &minLoc, &maxLoc);
> 
> ```

`blurredImageBGR[0]` is A, not B or R, right ? you have to adapt the indexing

---

<div class="post-metadata">

**Author:** ![Haider\_Saleem](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/haider_saleem/32/5847_2.png) [@Haider\_Saleem](https://forum.opencv.org/u/Haider_Saleem)\
**Post date:** [August 12, 2022, 10:29am UTC](https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823/3 "2022-08-12T10:29:25Z")

</div>

So issue starts from here

```auto
val orig = Mat(bmp.height, bmp.width, CvType.CV_8UC3)
    val myBitmap32 = bmp.copy(Bitmap.Config.ARGB_8888, true)

```

what could be an alternate solution for converting it to BRG then?

and for remaining c++, its actually written by c++ dev of our team and according to him it works fine as expected.

---

<div class="post-metadata">

**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\
**Post date:** [August 13, 2022, 8:16am UTC](https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823/4 "2022-08-13T08:16:25Z")

</div>

> [@Haider\_Saleem](#):
>
> what could be an alternate solution for converting it to BRG then?

so, problem is, you have an `argb` image on the java side and an expected `bgr` Mat on the c++ side.

maybe you can use mixChannels():

```auto

Mat argb2bgr(const Mat &m) {
	Mat _r(m.rows, m.cols, CV_8UC3);
	mixChannels({m}, {_r}, {1,2, 2,1, 3,0}); // drop a, swap r,b
    return _r;
}

```

you also dont need to split _anything_ into channels:

```auto

Mat dehaze(const Mat &argb, Scalar tw=Scalar(0.8, 0.8, 0.5)) {
    // convert to rgb, [0...1]
	Mat image = argb2bgr(argb);
	image.convertTo(image, CV_64F, 1.0/255);

	GaussianBlur(image, blurred, Size(41, 41), 30, 30);
    
    // each channel on its own row
    Mat channels = blurred.reshape(1,blurred.total()); 

    // check for max along x axis
    Mat maxVals;
    reduce(channels, maxVals, 0, REDUCE_MAX, image.depth());

    Mat tbgr = (Scalar::all(1.0) - blurred.mul(tw)) / maxVals;
}

```

btw, do you have a link to the algorithm you try to implement here ?

---

<div class="post-metadata">

**Author:** ![Haider\_Saleem](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/haider_saleem/32/5847_2.png) [@Haider\_Saleem](https://forum.opencv.org/u/Haider_Saleem)\
**Post date:** [August 15, 2022, 6:18am UTC](https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823/5 "2022-08-15T06:18:41Z")

</div>

I’ll try to implement your solution, and I guess the algo was written by developer himself.

Here’s complete source code.

```auto
int dehaze(Mat& image, Mat& result, double twBlue = 0.5, double twGreen = 0.5, double twRed = 0.1) {
    // MinMax function input placeholders
    double minVal;
    double maxVal;
    Point minLoc;
    Point maxLoc;
    // Image size placeholders
    int W;
    int H;
    // tresholding/rounding/clipping loop placeholders
    double valueBlue;
    double valueGreen;
    double valueRed;
    // intemediate array structure placehodlders
    Mat blurredImage;
    Mat A;
    Mat t;
    Mat dehazed;
    Mat dehazedSplit[3];
    Mat ABGR[3];
    Mat tBGR[3];
    Mat imageBGR[3];
    Mat blurredImageBGR[3];
    // Normalize, blur image and extract dimensions
    W = image.cols;
    H = image.rows;
    image.convertTo(image, CV_64FC3);
    image = image / Scalar(255, 255, 255);
    GaussianBlur(image, blurredImage, Size(41, 41), 30, 30);
    split(blurredImage, blurredImageBGR);

    // Estimate the A matrix
    A = Mat(H, W, CV_64FC3, Scalar(1, 1, 1));

    split(A, ABGR);
    minMaxLoc(blurredImageBGR[0], &minVal, &maxVal, &minLoc, &maxLoc);
    ABGR[0] = Scalar(maxVal);
    minMaxLoc(blurredImageBGR[1], &minVal, &maxVal, &minLoc, &maxLoc);
    ABGR[1] = Scalar(maxVal);
    minMaxLoc(blurredImageBGR[2], &minVal, &maxVal, &minLoc, &maxLoc);
    ABGR[2] = Scalar(maxVal);

    // Estimate the t matrix
    t = Mat(H, W, CV_64FC3, Scalar(0, 0, 0));
    split(t, tBGR);

    tBGR[0] = (Scalar(1) - blurredImageBGR[0].mul(Scalar(twBlue)) / ABGR[0]);
    tBGR[1] = (Scalar(1) - blurredImageBGR[1].mul(Scalar(twGreen)) / ABGR[1]);
    tBGR[2] = (Scalar(1) - blurredImageBGR[2].mul(Scalar(twRed)) / ABGR[2]);

    merge(tBGR, 3, t);
    merge(ABGR, 3, A);
    t.setTo(0.1, t < 0);

    // Estimate deahazed image
    dehazed = (image - A) / t + A;
    dehazed = dehazed.mul(Scalar(255, 255, 255));
    split(dehazed, dehazedSplit);
    split(image, imageBGR);

    // Clip and normalize image values
    for (int i = 0; i < H; i++) {
        for (int j = 0; j < W; j++) {
            valueBlue = abs(dehazedSplit[0].at<double>(i, j));
            valueGreen = abs(dehazedSplit[1].at<double>(i, j));
            valueRed = abs(dehazedSplit[2].at<double>(i, j));

            if (valueBlue > 255)
                valueBlue = 255;
            if (valueGreen > 255)
                valueGreen = 255;
            if (valueRed > 255)
                valueRed = 255;

            dehazedSplit[0].at<double>(i, j) = round(valueBlue);
            dehazedSplit[1].at<double>(i, j) = round(valueGreen);
            dehazedSplit[2].at<double>(i, j) = round(valueRed);
        }
    }

    merge(dehazedSplit, 3, dehazed);
    dehazed.convertTo(dehazed, CV_8UC3);
    result = dehazed;

    return 0;
}

```

---

<div class="post-metadata">

**Author:** ![Haider\_Saleem](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/haider_saleem/32/5847_2.png) [@Haider\_Saleem](https://forum.opencv.org/u/Haider_Saleem)\
**Post date:** [August 15, 2022, 8:40am UTC](https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823/6 "2022-08-15T08:40:16Z")

</div>

I implemented this code and it seems to be working with the provided algo, but only with high ram phones else causing heap issue on 4gb ram devices.

```auto

Mat argb2bgr(const Mat &m) {
	Mat _r(m.rows, m.cols, CV_8UC3);
	mixChannels({m}, {_r}, {1,2, 2,1, 3,0}); // drop a, swap r,b
    return _r;
}

```

and there’s another issue, the output mat when converted to bitmap, shows a red shade.

Here’s the output image.

 ![image](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/3/3e227dec4e5ea079c04f99d7f277e247b973904c.jpeg)

---

<div class="post-metadata">

**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\
**Post date:** [August 15, 2022, 8:45am UTC](https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823/7 "2022-08-15T08:45:19Z")

</div>

> [@Haider\_Saleem](#):
>
> the output mat when converted to bitmap, shows a red shade.

so, how do you convert it back ? (bgr vs rgb issue ?)

---

<div class="post-metadata">

**Author:** ![Haider\_Saleem](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/haider_saleem/32/5847_2.png) [@Haider\_Saleem](https://forum.opencv.org/u/Haider_Saleem)\
**Post date:** [August 15, 2022, 8:53am UTC](https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823/8 "2022-08-15T08:53:23Z")

</div>

I’m creating a bitmap like this

```auto
val outBmp = createBitmap(matOutput.width(),matOutput.height(),Bitmap.Config.RGB_565)

```

and then converting using

```auto
        Utils.matToBitmap(matOutput,outBmp)

```

---

<div class="post-metadata">

**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\
**Post date:** [August 15, 2022, 8:57am UTC](https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823/9 "2022-08-15T08:57:41Z")

</div>

try to convert to rgb before:

```
Imgproc.cvtColor(matOutput, matOutput, COLOR_BGR2RGB);

```

---

<div class="post-metadata">

**Author:** ![Haider\_Saleem](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/haider_saleem/32/5847_2.png) [@Haider\_Saleem](https://forum.opencv.org/u/Haider_Saleem)\
**Post date:** [August 15, 2022, 9:14am UTC](https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823/10 "2022-08-15T09:14:00Z")

</div>

So, I did this

```auto
        Imgproc.cvtColor(matOutput, matOutput, COLOR_BGR2RGB)
        val bmpOut = createBitmap(matOutput.width(), matOutput.height(), Bitmap.Config.RGB_565)

        Utils.matToBitmap(matOutput,bmpOut)

```

and now output is in blue shade

 ![image](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/9/96bc137a1235fd23f4e4a6935c9312e62e7e297c.jpeg)

---

<div class="post-metadata">

**Author:** ![Haider\_Saleem](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/haider_saleem/32/5847_2.png) [@Haider\_Saleem](https://forum.opencv.org/u/Haider_Saleem)\
**Post date:** [August 16, 2022, 12:46pm UTC](https://forum.opencv.org/t/camera2-bytearray-to-bgr-uint8-mat-conversion/9823/11 "2022-08-16T12:46:24Z")

</div>

Thanks Berak for all your help, dropping last channel worked for me and I guess while converting bitmap to mat it formed RGBA.

The solution:

```auto
    mixChannels({m}, {_r}, {0,0, 1,1, 2,2}); // drop a

```