Not able to read Video from Capture Card with GPU (Unsupported Format) while Reading with CPU works fine

cv::cuda::VideoReader uses the FFMPEG backend so if you can’t use cv::VideoCapture(src, CAP_FFMPEG) then it won’t work with cv::cuda::VideoReader.

That said I did manage to get the FFMPEG backend to work with direct show

you may be able to follow a similar procedure with V4L2 devices on linux but I coudn’t guarantee it. An alternative would be to write your own parser to read the raw input from your stereo cams by inheriting from cv::cudacodec::RawVideoSource.