I’ll say those pictures aren’t undistorted+rectified properly. they look like the optical axes aren’t yet parallel, which is what the rectification is supposed to do.
from these pictures, you’d get disparity in both directions (left and right). that’s… not what everything downstream assumes.
repeat your calibration. calibration is hard to get right. use charuco. always angle the board such that the board is not parallel to the image plane of either camera (not pointing at the camera, roughly).
and yes, leave that filter off for now. it obscures any problems.