StereoRectify : how to compensate a 4 pixel shift?

C++
, , ,
8

Thanks for your suggestions.

  1. For the intrinsic, I believe I can trust them (fixed optics) because it’s quite hard to modify them (compared to moving the 2 cameras (attached separately to the ROV) with respect to the other)

  2. I will double check, but I believe the calibration was done in water

  3. Yes, my main suspicion it that the camera moved with respect to each other between the acquisition and the calibration (other day, I think the ROV was retrieved from the see back to a ship, taken back to port, transported back to the university doing the tests, and I suppose put in some test pool to do the calibration).

For the 15 minutes video, I think the camera didn’t move (I haven’t noticed any change in the shift, at least nothing I noticed just looking at the values, so if there was a move, it should be <0.5 pixels). There are “no nice” features in the movies (it’s mainly small algae and rocks, with some turbidity and back-scattering of light on particles making the image a bit noisy) : so there are (nearly?) no features I could place to (sub) pixel accuracy by hand just by zooming the image (no sharp corners). On the other hand, there are hundreds (maybe even thousands) of “unsharp” features that could theoretically be matched between sucessive images. So maybe by averaging enough “not that good quality” features, it is still possible to get good results. Using goodFeaturesToMatch to find features and matching them with matchTemplate (TM_SQDIFF), I still manage to see this shift, so there should be enough information to correct it.

I will try playing with CY to see if I can improve things.

For the correspondance, for know it is goodFeaturesToMatch + matchTemplate. It should indeed be possible to use it to redo calibration (with a good amount of Ransac to eliminate the wrong matches)

EDIT : I tested to add 3.8 pixels to the CY of the right image : it solved the vertical shift (at least to a sub-pixel error that I no longer see “by hand” : I mainly get 0 shift, from time to time +1 or -1 pixels, in similar proportions)