Image conversion - Grab video at best quality

, ,
1

I am able to grab images and convert it into PixelType_BGR8packed as per OpenCV documentation examples given , if we try other options say like PixelType_BayerRG12, PixelType_BayerRG8 etc… these format says not supported by image converter.

End goal is to get the best possible fps and stream it, but PixelType_BGR8packed only supports a low frame rate, where if it is kept in say PixelType_BayerRG12, from Pylon viewer(GUI for camera), it can support upto 41fps

formatConverter.OutputPixelFormat = PixelType_BGR8packed; //Only supported format

IDE : Visual Studio 2019 Language : VC++
Image Libraries : Pylon 6 C++ Libs, OpenCV 4

try
{
    /* Get Instance of Camera */
    CInstantCamera camera(CTlFactory::GetInstance().CreateFirstDevice());

    cout << "Using Device :" << camera.GetDeviceInfo().GetModelName() << endl;

    GenApi::INodeMap& nodemap = camera.GetNodeMap();

    /*Open the available first Camera*/
    camera.Open();

    GenApi::CIntegerPtr width = nodemap.GetNode("Width");
    GenApi::CIntegerPtr height = nodemap.GetNode("Height");

    camera.MaxNumBuffer = 5;

    CImageFormatConverter formatConverter;


    /* Type of Output image format  PixelType_BGR12packed */
    formatConverter.OutputPixelFormat = PixelType_BGR8packed;
    // formatConverter.OutputBitAlignment = OutputBitAlignment_LsbAligned;


    CPylonImage pylonImage;

    int grabImages = 0;

    VideoWriter cvVideoCreator;
    Mat openCvImage;

    int counter = 0;

    /*Specify directory and file names*/
    std::string local_dir = "D:/OpenCV_Workspace/Recordings/";

    std::string videoFileName("FSV.avi");

    //Generic way of accessing
    cv::Size frameSize = Size((int)width->GetValue(), (int)height->GetValue());



    /* Type of Video format */
    cvVideoCreator.open(videoFileName, cv::VideoWriter::fourcc('M', 'J', 'P', 'G'), 15, frameSize, true);
    //Start grab and mention grabbing stratergy
    camera.StartGrabbing(c_countofImagesToGrab, GrabStrategy_LatestImageOnly);

    CGrabResultPtr ptrGrabResut;

    while (camera.IsGrabbing())
    {
        camera.RetrieveResult(5000, ptrGrabResut, TimeoutHandling_ThrowException);

        if (ptrGrabResut->GrabSucceeded())
        {
            cout << "Size X" << ptrGrabResut->GetWidth() << endl;
            cout << "Size Y" << ptrGrabResut->GetHeight() << endl;

            formatConverter.Convert(pylonImage, ptrGrabResut);

            openCvImage = cv::Mat(ptrGrabResut->GetHeight(), ptrGrabResut->GetWidth(), CV_8UC3, (uint8_t*)pylonImage.GetBuffer());

            if (captureImageandSave)
            {
                //Open a namedwindow
                namedWindow("Capture", WINDOW_NORMAL);
                //Publish an image to the specified window
                imshow("Capture", openCvImage);
                //ShowImage(pylonImage, "GrabbedPylon");

            }
            waitKey(1);
        }
    }
}

}

2

possibilities:

  • Basler camera can convert before passing to host. downside: USB/GigE bandwidth is limited.
  • Basler driver can convert before passing to application. potentially they have specifically optimized code for that.
  • OpenCV can convert (cvtColor) on CPU.
  • OpenCV can convert (cvtColor) on GPU using OpenCL
  • OpenCV can convert (cuda::cvtColor) on GPU using CUDA

the GPU options have all the usual caveats, namely that communicating with the GPU costs time (latency), so you have to know what you’re doing, or else it’s gonna be slower than using the CPU.

your best course of action is getting advice from Basler.