I wouldn’t worry about it. I only suggested the test as its a good way to check if things work on your system. As you managed to get it to work by changing the resolution there is no need.
For future reference it is probably because you haven’t added the environmental variable I mentioned.
export OPENCV_TEST_DATA_PATH=EXTRA_REPO/testdata/
Addtionaly your measuring the upload, download and createStereoSGM functions not just the disparity calculation so the speed up could be even greater. To be sure I would also remove Ptr<StereoSGBM> sgbm = StereoSGBM::create from your CPU timing and run some warm up calls to sgm->compute(left,right,disp); before starting your GPU timing.
I’m not sure if this is the case but it would be easy to find out by running a loop at different resolutions and querying the free memory using cuda::DeviceInfo::freeMemory().
Before anything else I would check the disparity results compared to the GPU. I worked on a project several years ago and at that time the GPU results were not as good and certain algorithms exhibited what looked like quantization artifacts.