Clarification on CALL_HAL Usage in pyrDown and Handling of ROIs in OpenCV

Shyama7004 · November 12, 2024, 10:52am

Hello OpenCV community,

I am working on understanding and potentially modifying the pyrDown function in OpenCV to ensure consistent behaviour when processing different regions of interest (ROIs). In particular, I’ve observed issues when pyrDown is applied to an ROI compared to when it’s applied to a full image and then cropped to the same ROI. This discrepancy appears when using certain border types and might relate to how the CALL_HAL macro is utilised.

Here’s the snippet of the code: (Line 1348 to 1378 of pyramid.cpp)

void cv::pyrDown( InputArray _src, OutputArray _dst, const Size& _dsz, int borderType )
{
CV_INSTRUMENT_REGION();

CV_Assert(borderType != BORDER_CONSTANT);

CV_OCL_RUN(_src.dims() <= 2 && _dst.isUMat(),
           ocl_pyrDown(_src, _dst, _dsz, borderType))

CV_OVX_RUN(_src.dims() <= 2,
           openvx_pyrDown(_src, _dst, _dsz, borderType))

Mat src = _src.getMat();
Size dsz = _dsz.empty() ? Size((src.cols + 1)/2, (src.rows + 1)/2) : _dsz;
_dst.create( dsz, src.type() );
Mat dst = _dst.getMat();
int depth = src.depth();

if(src.isSubmatrix() && !(borderType & BORDER_ISOLATED))
{
    Point ofs;
    Size wsz(src.cols, src.rows);
    src.locateROI( wsz, ofs );
    CALL_HAL(pyrDown, cv_hal_pyrdown_offset, src.data, src.step, src.cols, src.rows,
             dst.data, dst.step, dst.cols, dst.rows, depth, src.channels(),
             ofs.x, ofs.y, wsz.width - src.cols - ofs.x, wsz.height - src.rows - ofs.y, borderType & (~BORDER_ISOLATED));
}
else
{
    CALL_HAL(pyrDown, cv_hal_pyrdown, src.data, src.step, src.cols, src.rows, dst.data, dst.step, dst.cols, dst.rows, depth, src.channels(), borderType);
}

My questions are :

What role does CALL_HAL play in pyrDown, and how does it impact border handling and ROI consistency?
Are there specific considerations when passing ROI offsets and image sizes to CALL_HAL that might affect the outcome?
How do cv_hal_pyrdown and cv_hal_pyrdown_offset differ in their handling of ROIs and borders, and what precautions should be taken to ensure consistent results between full-image and ROI-based processing?
Are there best practices or examples for modifying pyrDown or similar functions to ensure uniform results across various border types and ROIs?

Any guidance on this or related experiences would be greatly appreciated.

sturkmen · November 14, 2024, 7:21pm

here is the ChatGPT explanation

The cv::pyrDown function is designed to downscale an image by a factor of 2, reducing its resolution. It uses Gaussian smoothing to prevent aliasing and has a structure that handles various types of input, optimizations, and low-level operations. Here’s a breakdown of the code:

Key Steps and Explanation

Instrumentation for Performance Monitoring:
```
CV_INSTRUMENT_REGION();
```
- This macro is used for profiling and debugging purposes. It marks the start of a region in the code, allowing OpenCV to track performance data for pyrDown.
Asserting Border Type:
```
CV_Assert(borderType != BORDER_CONSTANT);
```
- The function does not support BORDER_CONSTANT. This restriction ensures that only other border handling types (like BORDER_REFLECT, BORDER_REPLICATE) are used.
OpenCL Acceleration Check:
```
CV_OCL_RUN(_src.dims() <= 2 && _dst.isUMat(),
           ocl_pyrDown(_src, _dst, _dsz, borderType))
```
- If the source is a 2D matrix and the destination is a UMat, OpenCV attempts to use an OpenCL implementation (ocl_pyrDown) for GPU acceleration. If the conditions aren’t met or OpenCL fails, it proceeds with the CPU implementation.
Defining Source and Destination Matrices:
```
Mat src = _src.getMat();
Size dsz = _dsz.empty() ? Size((src.cols + 1) / 2, (src.rows + 1) / 2) : _dsz;
_dst.create(dsz, src.type());
Mat dst = _dst.getMat();
int depth = src.depth();
```
- Converts _src to a Mat for processing on the CPU.
- Computes the default destination size (dsz) as half of src’s dimensions (rounded up) if _dsz is empty.
- Initializes the destination matrix (dst) with the computed size and source type.

Handling Submatrix Scenarios:

if (src.isSubmatrix() && !(borderType & BORDER_ISOLATED)) {
    // ...
    CALL_HAL(pyrDown, cv_hal_pyrdown_offset, src.data, src.step, src.cols, src.rows,
             dst.data, dst.step, dst.cols, dst.rows, depth, src.channels(),
             ofs.x, ofs.y, wsz.width - src.cols - ofs.x, wsz.height - src.rows - ofs.y, borderType & (~BORDER_ISOLATED));
}

If src is a submatrix, meaning it’s part of a larger matrix, and the border type is not isolated, the function calculates the region of interest (ROI) of the source.
CALL_HAL attempts to use a hardware abstraction layer (HAL) function cv_hal_pyrdown_offset to optimize the downsampling operation.

Standard Downsampling Call:
```
else {
    CALL_HAL(pyrDown, cv_hal_pyrdown, src.data, src.step, src.cols, src.rows, dst.data, dst.step, dst.cols, dst.rows, depth, src.channels(), borderType);
}
```
- If the source matrix is not a submatrix, it calls cv_hal_pyrdown without any offsets, which is the standard HAL function for the downsampling.

Selecting the Downsampling Function:

PyrFunc func = 0;
if (depth == CV_8U)
    func = pyrDown_< FixPtCast<uchar, 8> >;
else if (depth == CV_16S)
    func = pyrDown_< FixPtCast<short, 8> >;
else if (depth == CV_16U)
    func = pyrDown_< FixPtCast<ushort, 8> >;
else if (depth == CV_32F)
    func = pyrDown_< FltCast<float, 8> >;
else if (depth == CV_64F)
    func = pyrDown_< FltCast<double, 8> >;
else
    CV_Error(cv::Error::StsUnsupportedFormat, "");

This block sets the appropriate pyrDown_ function based on the data type (depth) of src:
- FixPtCast<uchar, 8> and other FixPtCast templates handle fixed-point casting for integer types.
- FltCast<float, 8> and FltCast<double, 8> handle floating-point casting for float and double data types.
Each cast type ensures the appropriate data precision for downsampling.

Applying the Downsampling Function:
```
func(src, dst, borderType);
```
- Finally, the chosen function func is called to perform the downsampling operation with the specified border type.

Summary

This pyrDown function implementation supports both CPU and GPU acceleration (when using UMat), hardware abstraction optimizations via HAL, and data type handling for various depths. It achieves downsampling by combining Gaussian smoothing with rescaling, minimizing aliasing in the output.

sturkmen · November 14, 2024, 7:31pm

void cv::pyrDown( InputArray _src, OutputArray _dst, const Size& _dsz, int borderType )
{
    CV_INSTRUMENT_REGION();

    CV_Assert(borderType != BORDER_CONSTANT);

    CV_OCL_RUN(_src.dims() <= 2 && _dst.isUMat(),
               ocl_pyrDown(_src, _dst, _dsz, borderType))

    Mat src = _src.getMat();
    Size dsz = _dsz.empty() ? Size((src.cols + 1)/2, (src.rows + 1)/2) : _dsz;
    _dst.create( dsz, src.type() );
    Mat dst = _dst.getMat();
    int depth = src.depth();

    if(src.isSubmatrix() && !(borderType & BORDER_ISOLATED))
    {
        Point ofs;
        Size wsz(src.cols, src.rows);
        src.locateROI( wsz, ofs );
        CALL_HAL(pyrDown, cv_hal_pyrdown_offset, src.data, src.step, src.cols, src.rows,
                 dst.data, dst.step, dst.cols, dst.rows, depth, src.channels(),
                 ofs.x, ofs.y, wsz.width - src.cols - ofs.x, wsz.height - src.rows - ofs.y, borderType & (~BORDER_ISOLATED));
    }
    else
    {
        CALL_HAL(pyrDown, cv_hal_pyrdown, src.data, src.step, src.cols, src.rows, dst.data, dst.step, dst.cols, dst.rows, depth, src.channels(), borderType);
    }

    PyrFunc func = 0;
    if( depth == CV_8U )
        func = pyrDown_< FixPtCast<uchar, 8> >;
    else if( depth == CV_16S )
        func = pyrDown_< FixPtCast<short, 8> >;
    else if( depth == CV_16U )
        func = pyrDown_< FixPtCast<ushort, 8> >;
    else if( depth == CV_32F )
        func = pyrDown_< FltCast<float, 8> >;
    else if( depth == CV_64F )
        func = pyrDown_< FltCast<double, 8> >;
    else
        CV_Error( cv::Error::StsUnsupportedFormat, "" );

    func( src, dst, borderType );
}

actually CALL_HAL do nothing for now. because cv_hal_pyrdown and cv_hal_pyrdown_offset functions are not implemented yet.

inline int hal_ni_pyrdown(const uchar* src_data, size_t src_step, int src_width, int src_height, uchar* dst_data, size_t dst_step, int dst_width, int dst_height, int depth, int cn, int border_type) { return CV_HAL_ERROR_NOT_IMPLEMENTED; }

//! @cond IGNORED
#define cv_hal_pyrdown hal_ni_pyrdown
//! @endcond

inline int hal_ni_pyrdown_offset(const uchar* src_data, size_t src_step, int src_width, int src_height,
                                 uchar* dst_data, size_t dst_step, int dst_width, int dst_height,
                                 int depth, int cn, int margin_left, int margin_top, int margin_right, int margin_bottom, int border_type)
{ return CV_HAL_ERROR_NOT_IMPLEMENTED; }

//! @cond IGNORED
#define cv_hal_pyrdown_offset hal_ni_pyrdown_offset

Topic		Replies	Views
Unexpected result of pyrDown Python imgproc	0	330	January 7, 2022
Mismatch of result between cv::PyrUp/Down() and cv::cuda::PyrUp/Down() cuda	0	212	March 10, 2021
Is there a function in OpenCV that is similar to 'gen_measure_rectangle2' in halcon? metrology , halcon , imgproc	8	2417	April 6, 2021
Confused about OpenCL & dnn backends Python dnn , opencl	0	96	June 15, 2024
Motion detection by means of ROI grid Python	3	1871	December 30, 2020

Clarification on CALL_HAL Usage in pyrDown and Handling of ROIs in OpenCV

Key Steps and Explanation

Summary

Related topics