depends on what modules of OpenCV you use.
DNN stuff can use OpenCL, CUDA, intel inference engine (library), …
for a long time Raspberry PI GPUs hadn’t even been documented enough to allow decent open source drivers. it’s also not that powerful. according to some sources, the VideoCore IV has ~24 Gflop/s of processing power. that’s less than low end desktop CPUs deliver these days. you should absolutely expect anything to run like molasses on a raspberry pi, even if you get the GPU to do the work.
the best you can hope for is that someone made an OpenCL driver for it, and then OpenCV might be able to use it, but you should expect this to be a struggle.