# Wide-baseline stereo on circles grid, enormous stereo RMS

**URL:** <https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595>\
**Category:** Uncategorized\
**Tags:** calib3d, calib\
**Created:** [June 22, 2026, 8:24pm UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595 "2026-06-22T20:24:22Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![marisano](https://avatars.discourse-cdn.com/v4/letter/m/e0b2c6/32.png) [@marisano](https://forum.opencv.org/u/marisano)\
**Post date:** [June 22, 2026, 8:24pm UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595/1 "2026-06-22T20:24:22Z")

</div>

Thanks so much for providing this test code! I’ve used it as the basis of my own asymmetric circle grid stereo calibration code. Intrinsic camera parameter determination functions well (0.3 and 0.5 rms error), but when I try to achieve stereo calibration, things go horribly awry (rms of 708,000). I think this is because the left and image pairs are offset from one another by about 72°. However, apparently, fisheye lenses were not used. One constraint I have is that the data were collected before my time in the lab, and the test subjects for which the calibration is being done are not presently available. Moreover, the cameras themselves are not readily available, making even collecting new calibration images challenging.

Thus, I first wonder if you were able to get stereo calibration to work with an asymmetric circular grid pattern. If so, a set of images I could test my code on would be well-appreciated… Or more example source!

---

<div class="post-metadata">

**Author:** ![marisano](https://avatars.discourse-cdn.com/v4/letter/m/e0b2c6/32.png) [@marisano](https://forum.opencv.org/u/marisano)\
**Post date:** [June 22, 2026, 9:24pm UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595/2 "2026-06-22T21:24:46Z")

</div>

Interestingly, if I call cv2.cornerSubPix() after having called cv2.findCirclesGrid(), the rms error drops from 708,000 to 136. I tried including cornerSubPix() because I kept reading that it _may_ not be necessary when using circular grids. I added the following code snippet:

```python
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 50, 0.001)

print('Using pattern type asymmetric_circles')

ret_l, corners_l = cv2.findCirclesGrid(grayL, pattern_size, flags=cv2.CALIB_CB_ASYMMETRIC_GRID)
ret_r, corners_r = cv2.findCirclesGrid(grayR, pattern_size, flags=cv2.CALIB_CB_ASYMMETRIC_GRID)

if ret_l and ret_r:
    corners_l = cv2.cornerSubPix(grayL, corners_l, (5, 5), (-1, -1), criteria)
    corners_r = cv2.cornerSubPix(grayR, corners_r, (5, 5), (-1, -1), criteria)

```

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [June 22, 2026, 9:33pm UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595/3 "2026-06-22T21:33:50Z")

</div>

> [@marisano](#):
>
> the left and image pairs are offset from one another by about 72°

that’s an angle, not a distance. are you saying the cameras are “going cross-eyed”?

I think the literature counts that as “wide baseline”. that situation has unique challenges.

With OpenCV 5.0 they released algorithms for _multi-camera_ calibration. Perhaps you should look into that. If you are working with AI, you may have to tell it to use [`docs.opencv.org/5.0/`](https://docs.opencv.org/5.0/) explicitly rather than using Google search. Google somehow deindexed the entire docs domain.

One typical issue with calibration is that the patterns are recognized upside-down, or whatever causes the points on the pattern to be _enumerated_ backwards. This can happen very easily if the canonical orientation is close to some notion of “zero degrees” or wrap-around.

`cornerSubPix()` is not for circles. Circles have no corners/saddle point in their center. Not just is it “not necessary”, it CANNOT WORK on circles. If you plot the picture along with the points _after_ `cornerSubPix` you should see the points being clearly off-center.

---

<div class="post-metadata">

**Author:** ![marisano](https://avatars.discourse-cdn.com/v4/letter/m/e0b2c6/32.png) [@marisano](https://forum.opencv.org/u/marisano)\
**Post date:** [June 23, 2026, 6:48am UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595/4 "2026-06-23T06:48:55Z")

</div>

Thanks for the feedback!

> are you saying the cameras are “going cross-eyed”?

I suppose they were. I don’t actually know the cameras they were oriented. I just inherited the resultant videos. To get one of the images to best match the other, I need to rotate it about 72°. I have seen the documentation for the multi-camera calibration under the now released OpenCV 5.0., but I wanted to be sure what I’m working with is a good foundation before going that route (also, when I first looked it up, it hadn’t been released yet). The set-up actually used four cameras, but two of them work at a lower resolution (1280x720, rather than 1920x1080) and frame rate (500, rather than 1000). But yes, I absolutely do intend to explore that code soon.

> [@crackwitz](#):
>
> One typical issue with calibration is that the patterns are recognized upside-down, or whatever causes the points on the pattern to be _enumerated_ backwards. This can happen very easily if the canonical orientation is close to some notion of “zero degrees” or wrap-around.

I’ve also read of that occurring, but I don’t see how it would, since the pattern coloration and arrow ordering demonstrate the start and enumeration order. I can double check that it is properly order in all of the calibration image pairs.

Concerning `cornerSubPix()`, the following did come from `geeksforgeeks.org` (`https://www.geeksforgeeks.org/machine-learning/calibratecamera-opencv-in-python/`), but notice the comment at the end of this source code blurb.

```python
# If true is returned, 
# then 3D and 2D vector points are updated and corner is drawn on image
if ret == True:
    obj_points.append(obj3d)

    corners2 = cv.cornerSubPix(gray, corners, (11, 11), (-1, -1), criteria)
    # In case of circular grids, 
    # the cornerSubPix() is not always needed, so alternative method is:
    # corners2 = corners
    img_points.append(corners2)

```

That said, it appears that the actual improvement came from enforcing a criteria (of `(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 50, 0.001)`, rather than `(None, 0, 0)`), which, apart from within `cornerSubPix()`, is finally applied in `cv2.stereoCalibrate()`. However, in this case, calling `cornerSubPix()` produced absolutely no ill effect, as can be seen in the attached image, which was displayed and saved after having called `cornerSubPix()`. Moreover, the reprojection error is exactly the same in either case: 136.07265517186306. [I now realize it doesn’t strictly make sense to call a corner enhancing function on a circular pattern, but then, neither does calling `drawChessboardCorners()`. I imagined perhaps each result of the iterative process was reversed.]

 ![Detected Pattern left 1](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/2/2e7a23ba696e0dbc7871e9cedb02a8f5fc882be6.jpeg)

(I know the pattern’s tiny! I didn’t collect these…)

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [June 23, 2026, 7:06am UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595/5 "2026-06-23T07:06:39Z")

</div>

> [@marisano](#):
>
> To get one of the images to best match the other, I need to rotate it about 72°

please just show some pictures. human prose is too inaccurate.

> [@marisano](#):
>
> Concerning `cornerSubPix()`, the following did come from `geeksforgeeks.org` (`https://www.geeksforgeeks.org/machine-learning/calibratecamera-opencv-in-python/`), but notice the comment at the end of this source code blurb.

That is a SEO junk site. _Never_ trust it. Always assume they are maliciously trying to mislead you. Correctness, accuracy, truth don’t factor into what they post. Their only goal is to rank at the top in Google results, for the advertising money.

> [@geeksforgeeks](#):
>
> ```auto
> # In case of circular grids, 
> # the cornerSubPix() is not always needed, so alternative method is:
> # corners2 = corners
> 
> ```

That comment in that code proves they are a _profoundly incompetent_ lot. I’d have much stronger words to describe them.

> [@marisano](#):
>
> `(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 50, 0.001)`, rather than `(None, 0, 0)`

what do you mean? _broken_ termcrits (the second expression) _must_ cause everything to fail. _sensible_ termcrits are _required_.

> [@marisano](#):
>
> absolutely no ill effect

_please_ don’t think like that. the scientific term for that is “magical thinking”, the same fallacy that underpins homeopathy and cargoculting. _do not_ just do things because they have “no ill effect”. in that instance, “no ill effect” was _its **failure** to do its job_, and you are _lucky_ it failed gracefully (it returned the input points). if it had found _something_ in the image to interpret as a corner, which would likely have been the _perimeter_ of a circle, you’d get completely ruined points instead. you decide if your code should rely on luck to not fail catastrophically. and you must _understand_ what a thing does and how it does that, how it can fail, when to use it (appropriate use) and when _not_ (inappropriate use).

> [@marisano](#):
>
> I know the pattern’s tiny!

it is. and that fact makes the entire picture useless for intrinsic or extrinsic calibration. it shows no “foreshortening”, but that is the property that is _crucial_ to calibration.

---

<div class="post-metadata">

**Author:** ![marisano](https://avatars.discourse-cdn.com/v4/letter/m/e0b2c6/32.png) [@marisano](https://forum.opencv.org/u/marisano)\
**Post date:** [June 23, 2026, 6:58pm UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595/6 "2026-06-23T18:58:43Z")

</div>

Thanks for the great feedback!

> [@crackwitz](#):
>
> That comment in that code proves they are a _profoundly incompetent_ lot. I’d have much stronger words to describe them.

I understand what you’re saying. But it is confusing that we’re discussing corners at all when applying calibration to circular grids. This makes it difficult to determine what points are necessarily corners (or should be), and what points aren’t. Perhaps wrapper functions would be useful…

> [@crackwitz](#):
>
> > [@marisano](#):
> >
> > `(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 50, 0.001)`, rather than `(None, 0, 0)`
> 
> what do you mean? _broken_ termcrits (the second expression) _must_ cause everything to fail. _sensible_ termcrits are _required_.

I don’t exactly know what I mean yet. I’m learning as I go along. I thought that if I had no criteria, then there would be no max iterations and no cutoff. (It appears that the documentation for OpenCV 5 is much cleaner. However, for previous versions, it’s been very difficult for me to follow. There are several different ones, and I often don’t know what’s been superseded or precisely where to go to learn from competent sources.)

> [@crackwitz](#):
>
> > [@marisano](#):
> >
> > absolutely no ill effect
> 
> _please_ don’t think like that. the scientific term for that is “magical thinking”, the same fallacy that underpins homeopathy and cargoculting. _do not_ just do things because they have “no ill effect”. in that instance, “no ill effect” was _its **failure** to do its job_, and you are _lucky_ it failed gracefully (it returned the input points).

Very poor wording on my part. How about, ‘no observable ill effects’? Also, I didn’t mean in general; I only meant in that usage case. That is, from having called `cornerSubPix()`in this scenario, I couldn’t determine that anything had gone awry. Also, because the results improved dramatically, I initially concluded that that function call was responsible for the improvement. I would say that I was extremely _ **unlucky** _ that the function failed gracefully. It would have been much better for it to have failed catastrophically, because then I could have detected the error myself, rather than imagining that there was something funny going on in the function (because really, corners ought to be corners).

I suppose that in the absence of known documentation, I could have looked at the source code. But I’m still not used to source code being open. I’m used to opaque APIs, where the documentation tells you what you need to know. So, I sought out documentation.

> [@crackwitz](#):
>
> > [@marisano](#):
> >
> > I know the pattern’s tiny!
> 
> it is. and that fact makes the entire picture useless for intrinsic or extrinsic calibration. it shows no “foreshortening”, but that is the property that is _crucial_ to calibration.

Thanks again! However, I wouldn’t know this straight off—or anyway, didn’t. I thought that it might also depend on the initial resolution. I also obtained intrinsic results that were about 0.57 for the left and 0.32 for the right camera, which, being well below 1, I considered fine for preliminary investigation. I also expected that the other two cameras ought to help considerably because, for one thing, they didn’t view the image (however tiny) from the same orientation, so their amounts of foreshortening are likely to be different.

Additionally, concerning foreshortening, I wasn’t sure it was necessary at all because of this former error: [findCirclesGrid can’t detect some asymmetric grids](https://github.com/opencv/opencv/issues/24971), as in, what was the code doing before this bug was fixed? So, I thought perhaps the distance from the cameras (as estimated by circle size) was being used to obtain such information. Also, in [Calibration Patterns Explained](https://calib.io/blogs/knowledge-base/calibration-patterns-explained), calib․io claims that both the elliptical and projected circle center need to be accounted for, and that OpenCV does neither. Is this still true in OpenCV 5? If so, how much work would it take to address? (I may not know anything now, but I’ve felt the pain, and might want to take a good crack at that later, if the need remains.)

As for the lack of video resolution, I thought that this would be problematic, albeit not necessarily for calibration. Instead, I thought that I’d have no hope of determining the head versus body position accurately, unless I labeled each frame of interest manually. Thus, I came here from automatic animal pose estimation, so that I might extract 3D coordinates and use those to help determine if that is indeed the case, and that software does not support circle grids.

Finally, I’m unable to attach multiple images because I’m a new user: “An error occurred: Sorry, new users can only put one embedded media item in a post.” So, here’s the right counterpart of the left image I posted earlier, to demonstrate what I mean by circa 72° rotation:

 ![Detected Pattern right 1](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/7/7a2a0707f5bccffd40c929fa5d19fcce9fc47f51.jpeg)

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [June 24, 2026, 11:20am UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595/7 "2026-06-24T11:20:26Z")

</div>

Re: “no ill effect” and the thinking I suspected, my communication style tends to “ramp up” if I get a whiff of certain things. That is to get ahead and in front of certain patterns, even before there is evidence of it, because of my own impatience and the sadly high probability of the evidence coming later. I meant no offense.

I am 100% with you that the docs have been lacking. v5.0 docs are said to have received some improvements. I don’t know yet what those are. v5.0 being so new, I haven’t worked with these docs yet, can’t judge them. Regardless, the docs deserve more improvement, of the “manual work and thought” kind, not so much the styling and presentation kind.

Points and corners, and how that relates to circles, it’s complicated. Usually you can take “corners” as a synonym for “points” and vice versa. There are very few instances where corners actually means _corners_, and `cornerSubPix` is one such situation. _Circles_ as a calibration grid is an uncommon flavor of “points”. Corners are typically “saddle points” in the image, i.e. checkerboard corners. These can be located at greatest accuracy (as done by `cornerSubPix`).

> [@marisano](#):
>
> Also, in [Calibration Patterns Explained](https://calib.io/blogs/knowledge-base/calibration-patterns-explained), calib․io claims that both the elliptical and projected circle center need to be accounted for, and that OpenCV does neither. Is this still true in OpenCV 5? If so, how much work would it take to address? (I may not know anything now, but I’ve felt the pain, and might want to take a good crack at that later, if the need remains.)

The issue with circles in 3D is that under perspective projection to 2D, the apparent center of the projected circle, which is an ellipse, _deviates_ from the projected center of the 3D circle. Note the order and meaning of the words. This is a geometric fact and makes circle grids actually _terrible_ for calibration. The more out-of-plane the circle presents, the greater this difference, but for calibration you WANT out-of-plane rotation, for the perspective foreshortening. Circles’ only real advantage is that in some situations, circles are easier to _make_ than checkerboard corners. Another reason is that sometimes your camera is so bad that you’re already struggling to see circle-shaped blobs, and can’t resolve corners at all. That has been the case with a thermal infrared camera I got to use many years ago.

So no, AFAIK opencv does not deal with this. It might, but I’m unaware of this. Dealing with this is difficult because compensation requires knowing the out-of-plane rotation. I doubt it can be solved in _closed form_. it probably requires iteration, at least a few iterations.

Note that even [calib.io](http://calib.io) only claims “more” accuracy for their own solution, but not that they’re getting unbiased results.

Circles grids are not made for accuracy. I’ve seen some _companies_ recommend circles grids and sell circles grids for their own systems. IDK what they were thinking. Perhaps not much.

Some geometric considerations can probably lead to unbiased results for circles, but I would simply not bother and just go with saddle points.

If you can do your calibration with software that supports circles grids, you should be able to transfer the calibration to other systems. It might require some calculations to adapt the matrix and distortion coefficients.

> [@marisano](#):
>
> Additionally, concerning foreshortening, I wasn’t sure it was necessary at all because of this former error: [findCirclesGrid can’t detect some asymmetric grids](https://github.com/opencv/opencv/issues/24971), as in, what was the code doing before this bug was fixed?

I have no idea how foreshortening should relate to that bug or to finding the grid. Looks like it “simply” failed to enumerate the pattern’s points correctly, and failed entirely in some cases where “recovering the grid” was made sufficiently more difficult by perspective.

By the way, I’ve been meaning to contribute a much more generic, robust, and flexible “grid recovery” algorithm. That would make “charuco” boards entirely moot. the only reason someone came up with “charuco” boards is that the usual checkerboard detection code FAILS if the pattern is _incomplete_. that is a failing of the detection code, not a failing of the good old (and robust) checkerboard pattern.

Thanks for the second picture. That’s approximately the situation I imagined.

The pattern is enumerated the same in both views. I have no real hypothesis as to why repro error should be that high. The solver must have converged to a wildly implausible solution. Most likely that is due to the patterns not covering the entire view, thus yielding distortion coefficients that satisfy the patterns, but show great distortion towards the edges of the views.

Can you share the camera matrix and distortion coefficients?

---

<div class="post-metadata">

**Author:** ![marisano](https://avatars.discourse-cdn.com/v4/letter/m/e0b2c6/32.png) [@marisano](https://forum.opencv.org/u/marisano)\
**Post date:** [June 25, 2026, 4:55am UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595/8 "2026-06-25T04:55:31Z")

</div>

> [@crackwitz](#):
>
> I meant no offense.

None taken. I fear I may sometimes do likewise, but without ever noticing. At any rate, I don’t mind a little butt-kicking to gain good answers. Totally worth it! Sometimes, I don’t know how else I would obtain so much information.

> [@crackwitz](#):
>
> I am 100% with you that the docs have been lacking. v5.0 docs are said to have received some improvements.

I just installed OpenCV 5 via conda (miniforge), so I’ll give it a go and have a look at its documentation, which does seem to be a lot cleaner. [Unfortunately however, AFAIK, none of the pre-compiled opencv libraries available via conda include the contrib packages, and definitely not CUDA support. I thought of building OpenCV 5, but was only able to retrieve the CPU-only source (although I didn’t attempt to retrieve the source via conda). Lack of CUDA isn’t important to me now, especially since the new engine is still CPU-only, but I may need some contrib packages.]

> [@crackwitz](#):
>
> There are very few instances where corners actually means _corners_, and `cornerSubPix` is one such situation.

Once you’d explicitly pointed it out, that made a lot of sense, because I had looked up Harris Corner Detection, and one can see that `cornerSubPix()` doesn’t have a parameter to specify that it’s receiving a circle, so it couldn’t possibly satisfy opposing criteria. I just hadn’t put the two together.

> [@crackwitz](#):
>
> sometimes your camera is so bad that you’re already struggling to see circle-shaped blobs, and can’t resolve corners at all. That has been the case with a thermal infrared camera I got to use many years ago.

Yes… I’d read about application of asymmetric circle grids to thermal cameras, but I’d forgotten how terrible their resolution used to be, so, although I didn’t collect the data or the corresponding calibration, I certainly wasn’t thinking of not being able to resolve corners at all as when this grid class should be used. I was only thinking of much more precise animal pose recognition, but I guess in the case of thermal cameras (etc.), you use whatever you can get.

> [@crackwitz](#):
>
> Note that even [calib.io](http://calib.io) only claims “more” accuracy for their own solution, but not that they’re getting unbiased results.

I do not understand bias in this context. Where should I go to read more about it? How can you know that your biased result is more accurate, since the bias itself may be inaccurate?

> [@crackwitz](#):
>
> I’ve seen some _companies_ recommend circles grids and sell circles grids for their own systems.

I have too…

> [@crackwitz](#):
>
> The issue with circles in 3D is that under perspective projection to 2D, the apparent center of the projected circle, which is an ellipse, _deviates_ from the projected center of the 3D circle.

> [@crackwitz](#):
>
> This is a geometric fact and makes circle grids actually _terrible_ for calibration.

**This**. OpenCV should be trumpeting this point, especially since there is so much known fallacious information concerning it out there. Even as support for circle grids is expanded, it should be clearly stated that this grid form is to be avoided, except in the rare thermal case you mentioned above (or if, for some reason, you can only obtain a very limited number of calibration images). At least until or unless the corrections you mention may apply are also provided or otherwise easily available.

> [@crackwitz](#):
>
> Some geometric considerations can probably lead to unbiased results for circles, but I would simply not bother and just go with saddle points.

If I had other data, I might not bother either (or would only do so to gain greater insight). But so far, I don’t.

> [@crackwitz](#):
>
> If you can do your calibration with software that supports circles grids, you should be able to transfer the calibration to other systems. It might require some calculations to adapt the matrix and distortion coefficients.

Well, if I can, it would be from OpenCV. For my use case, you already outperform MATLAB, and I don’t want to pony up buckage for opaque software that might provide better results, which I nonetheless expect would still be insufficient. However, I don’t understand what you mean by transferring the calibration to other systems. What do you mean by “other systems”? I thought that as long as the relative positions of the cameras remained consistent, and so did the cameras themselves and their lenses, then the same extrinsic and intrinsic calibration applies, only that more or less refined parameters could be obtained from a different kind of grid, or from more challenging collection circumstances, in the case of partial occlusion.

> [@crackwitz](#):
>
> I have no idea how foreshortening should relate to that bug or to finding the grid. Looks like it “simply” failed to enumerate the pattern’s points correctly, and failed entirely in some cases where “recovering the grid” was made sufficiently more difficult by perspective.

Isn’t foreshortening a form of perspective?

> [@crackwitz](#):
>
> By the way, I’ve been meaning to contribute a much more generic, robust, and flexible “grid recovery” algorithm.

Any idea of when that contribution might make its debut? I’m for supporting as many grids to the highest degree as feasible, because some data can be hard to obtain, and then you’ve only got the calibration grid it was collected with.

> [@crackwitz](#):
>
> The solver must have converged to a wildly implausible solution. Most likely that is due to the patterns not covering the entire view

They certainly don’t cover the entire view…

Here are the camera matrices and distortion coefficients:

Left camera intrinsic parameters:  
`[[57711.39149507121, 0.0, 969.4391598343224], [0.0, 18084.305855652063, 517.4350617464842], [0.0, 0.0, 1.0]]`

Left camera distortion coefficients:  
`[38.26993895418094, -6972.758752951257, 1.0781937674228974, -0.0954462285924587, -37.8133092773903]`

Right camera intrinsic parameters:  
`[[16228.966150748682, 0.0, 957.1090488270004], [0.0, 15085.707001085935, 537.1123617941882], [0.0, 0.0, 1.0]]`

Right camera distortion coefficients:  
`[3.4618162172099005, 222.28266616111136, 0.08763855453306493, -0.25452855984845424, 0.6366740321843442]`

Hmm. It is somewhat disturbing that the focal length is so different between the two camera, given that I think they were both using the same (unknown) fixed camera lens…

When `cv2.stereoCalibrate()`is called with `flags=cv2.CALIB_USE_INTRINSIC_GUESS`, rather than `cv2.CALIB_FIX_INTRINSIC`, the resultant camera matrices and distortion coefficients are the following:

Left camera intrinsic parameters (possibly updated):  
`[[43713.95149911353, 0.0, -302.44767813632194], [0.0, 10086.411839574377, 113.29474964910439], [0.0, 0.0, 1.0]]`

Left camera distortion coefficients (possibly updated):  
`[309.93970861885214, -9791.178443208812, -0.5149032842527181, -9.139777025342118, -11239.073080795077]`

Right camera intrinsic parameters (possibly updated):  
`[[44086.414339342926, 0.0, -475.78297501653327], [0.0, 17768.421615711402, 124.13615595948114], [0.0, 0.0, 1.0]]`

Right camera distortion coefficients (possibly updated):  
`[-5.228133532870598, 162.34703439719462, 1.7975605314388026, 0.6821279979630578, -429.706811944283]`

Moreover, the reprojection error dropped from 136.073 to 69.470.

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [June 25, 2026, 1:04pm UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595/9 "2026-06-25T13:04:20Z")

</div>

> [@marisano](#):
>
> I do not understand bias in this context.

In this context, the difference between the center of the ellipse (projected circle), and the 3D circle’s projected center, always deviates in a particular direction that directly relates to which way the circle faces, relative to the camera’s image plane. I’ll call that “bias”. if the calib pattern is parallel to the camera’s sensor plane, all the circles project straight, and the centers coincide. no bias. if the pattern faces away from the camera in any particular direction, then all the ellipse centers deviate from the projected circle centers _the same way_. if you were to take these ellipse centers as truth, then your estimation would be biased. the individual biases wouldn’t “cancel out”.

“bias” is a general term for something that is “off the mark” in some way. when speaking of probability distributions, or populations, then it’s a term you can use to describe when the mean (a population statistic, not just individual samples) deviates from some value that you expect it to have. let’s say we’re in some factory and they are filling bottles with water. if their machine fills bottles, and we take a statistic over a day’s production, and that happens to be distributed _normally_ with µ=1.467 L, with whatever σ (it’s mostly irrelevant), then it would be fair to say the machine is biased negatively because it’s differing from the 1.50 L mark by (2.2%) more than someone deems acceptable (say 1%).

> [@marisano](#):
>
> I don’t understand what you mean by transferring the calibration to other systems.

IIRC, at least at some point in the past, the calibration results from the MATLAB calibration toolbox were using a lens model that was _normalized_ differently from how OpenCV does it. either the one or the other expressed the distortion coefficients independent of sensor resolution, while the other, respectively, had a distortion model formulated to take straight image space coordinates. or something like that. can easy to convert, if the distortion models are clear.

other example: if one program had a _distortion_ model, that means the model takes true points, and distorts them into “lens-distorted points”. if another program had an _undistortion_ model, its equation would do the opposite. since those models might be polynomials or something similar, they are costly to _invert_, and may not even be invertible in the general case, only if the lens is sufficiently “tame”. then, converting between those models takes some effort and numerical iteration.

> [@marisano](#):
>
> Any idea of when that contribution might make its debut?

that is contingent on my being bored enough and un-stressed enough to have a clear head for this kind of stuff. unless some company decides that they want me to work on it, I estimate the probability to be infinitesimal.

the idea is really not that complex. it’s just geometry, more or less. pick a point, pick its nearest neighbor, assume that you can walk the grid by taking the p\_1-p\_0 vector and adding it to p\_1, and then look for the nearest point to _that_, and you should have found p\_2. next step, maybe go one order higher with the differences, so you can predict _curvature_ (or perspective distortion).  
and that’s grid recovery along one dimension.  
for the second dimension, either assume a rectangular grid, give p\_1-p\_0 a 90 degree rotation (trivial in 2D), add it, look for the nearest point, and do the same thing along that direction. _or_, from p\_0, pick whatever remaining points are about as close as p\_1 was, and walk them. this could result in the recovery of triangular/hexagonal grids. if some assumption of the grid type can be made (say, it’s square), that would take much guesswork out of it and make it more robust.  
with some logic, this should be able to jump over missing points, or walk around them. It’s really just “flood fill/region growing”, except you aren’t on a square grid, but you have to discover it as you go.  
this idea could, given local guidance (not looking for points, but the centers of “modules”/squares), even recover warped QR codes and similar things.

> [@marisano](#):
>
> Here are the camera matrices and distortion coefficients:

the two focal lengths of the _left_ camera matrix being so much different (57.7e3, 18e3) worries me greatly. that would imply a non-square image sensor, but the image does not show that.

at least it seems to have figured out the cx, cy = 969, 517, which are plausible for a FullHD camera.

the distortion coefficients are absolutely wild. usually, the later coefficients go wild a little, that’s not much of a problem, but your k1, k2 are already way out of any reasonable range. the images you presented show no noticeable lens distortion, so any DCs should be very much smaller than 1.

the right camera’s matrix looks better, but still the foci shouldn’t differ by 7.5%. they should differ by something less than 1%, less than 0.1% even. the DCs are still too wild.

if those cameras are supposed to be of the same brand and model, their calibrations should be much more similar to each other than these two are.

`USE_INTRINSIC_GUESS` does not constrain the calibration, but it lets you start from a guess. giving that can help make convergence go faster, or hit the right _local optimum_, if you know what it _should_ be, from theoretical considerations.

FIXING the intrinsics _only_ makes sense if you already did intrinsic calibrations for each camera individually, and you think those are better than what the stereo calibration could calculate.

repro errors on that order mean the data is bad. repro error, for good calib data, should be below 1, meaning it is good enough to explain pictures to an accuracy of better than one pixel. some systems are so high resolution, and/or have such wide field of view, that the repro error may rise above one pixel, but it shouldn’t rise above, say, 3-10 pixels, and if it did, I would recommend thinking about whether the distortion model is too simple to model the lens, or whether the calibration data is somehow flawed.

my opinion: bad calibration data. and that is _not_ due to circles, but due to insufficiently covering the view.

to be able to _confidently_ assert that, I would have to see the entire data set.

what I’ve seen so far is _one_ pair of pictures, and I wouldn’t want to judge from N=1, but if that pair is representative…

bad calibration data is usually _not_ salvageable.

sometimes it _might_ be, if there are image pairs that could be called “outliers”, and all other pairs are fine, then you could kick such a bad pair out of the set.

---

<div class="post-metadata">

**Author:** ![marisano](https://avatars.discourse-cdn.com/v4/letter/m/e0b2c6/32.png) [@marisano](https://forum.opencv.org/u/marisano)\
**Post date:** [June 26, 2026, 5:19am UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595/10 "2026-06-26T05:19:38Z")

</div>

> [@crackwitz](#):
>
> the two focal lengths of the _left_ camera matrix being so much different (57.7e3, 18e3) worries me greatly. that would imply a non-square image sensor, but the image does not show that.

Good point! I didn’t consider _fy_ because I assumed the pixels are square (they are). I preferred the left camera matrix because a) I thought its _fx_ focal length made more sense; b) I don’t understand distortion coefficients yet; and c) when I tried only supplying `stereoCalibrate()` with one pre-solved camera matrix and set of distortion coefficients (from `cameraCalibrate()`) (the corresponding parameters for the other camera having been zeroed out), I got a lower rms when the left camera matrix and distortion coefficients were provided. The two cameras are definitely the same make and model.

> [@crackwitz](#):
>
> I would recommend thinking about whether the distortion model is too simple to model the lens, or whether the calibration data is somehow flawed.

I’ll look into the distortion model—especially now that I have finally found reliable documentation (LLMs invading search is often really not so helpful; they’re could be good for parsing questions, but not providing reliable answers)—but I’m almost certain the real problem is bad calibration data due to insufficient view coverage.

AFAIK, because I’m a new forum member, I’m still limited to attaching a single image per post; if that can be alleviated, I can post the others. But either way, I can assure you that a large portion of the field of view is not covered in the calibration videos. However, here is something else that might also play a role:

So far, I’ve used the same images for single camera calibration as for stereo calibration. However, this seems overly restrictive, in that sometimes one camera may have a complete view of the entire grid, while the other does not. This would be fine for use in calibrating the one camera, and could allow for greater coverage, but would fail for stereo calibration. Later today, I’ll add another folder where images can be placed for single camera calibration, alongside those used for stereo calibration.

Additionally, there were four cameras used to collect the data. If the totality of represented frames from them all constitute sufficient coverage (I don’t know if they do or not; I just chose the pair with highest resolution because I knew I’d need it for improved pose estimation), would using `calibrateMultiview()` be able to salvage the calibration?

Finally, what is the advantage of having resolution below half a pixel? (I can imagine roundoff error affecting calculations with \>0.5 pixel resolution.) Does it come into play when one camera has higher resolution than another? Could one, for example, aim with greater precision than their actual vision by means of such resolution?

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [June 26, 2026, 9:40am UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595/11 "2026-06-26T09:40:31Z")

</div>

> [@marisano](#):
>
> I’ll look into the distortion model

don’t do that, unless you mean to understand the standard model more deeply. my suggestion was a hypothetical for situations other than yours. the standard distortion model is fine. your issue is that the calibration data was gathered wrong, and perhaps the “experimental setup” might be flawed.

yes, the requirement of _overlap_ for a stereo calibration excludes images that are useful for intrinsic calibration. that is also why stereo calibration functions support _using_ individual intrinsic calibration results during stereo calib.

> [@marisano](#):
>
> calibrateMultiview()

that’s a new function coming with v5.0. it _should_ be worth checking out. I have never used that before and don’t know its requirements.

> [@marisano](#):
>
> Finally, what is the advantage of having resolution below half a pixel? (I can imagine roundoff error affecting calculations with \>0.5 pixel resolution.) Does it come into play when one camera has higher resolution than another? Could one, for example, aim with greater precision than their actual vision by means of such resolution?

you ask questions that take a lot to answer.

first, resolution _defines_ what a pixel is. it is not expressed in pixels. I’ll assume you meant accuracy, or maybe reprojection error.  
since you are dealing with stereo vision, accuracy matters a lot. an appreciable amount of effort goes into squeezing accuracy from the data.  
you haven’t said “disparity map” yet, so I can’t assume or go into that math. a “pixel” is only a thing because our images are discrete things (image sensors discretize an area of light into pixels). if they weren’t discrete things, we’d instead talk about angles or tangent values. the more precisely you can express angles, e.g. of “sight rays”, one from each eye, that intersect the same physical point, the more precisely you can calculate the _distance_ to that point. the closer the point is to your eyes/cameras, the more stable is that result. the further away, the more inaccurate it gets. equal triangles and all that.  
algorithms to match one image to another, for the purposes of stereo vision, operate by matching _blocks_, i.e. small image patches. that allows matching at subpixel resolution. OpenCV’s stereo matcher algorithms typically express their disparity values with 4 bits of subpixel resolution, or 1/16 of a pixel.  
two cameras having differing resolutions don’t matter all that much. if you go the “_stereo_ vision with disparity map” path, at some point the images get “rectified”, which applies camera matrix and lens distortion model in such a way that you get two new pictures that show map “epipolar lines” to be parallel (and horizontal). that’s a requirement for block matching. that step also equalizes the resolutions of both cameras, so either one image gets blown up, or the other’s “excess” resolution gets thrown away.

---

<div class="post-metadata">

**Author:** ![marisano](https://avatars.discourse-cdn.com/v4/letter/m/e0b2c6/32.png) [@marisano](https://forum.opencv.org/u/marisano)\
**Post date:** [June 26, 2026, 11:48am UTC](https://forum.opencv.org/t/wide-baseline-stereo-on-circles-grid-enormous-stereo-rms/24595/12 "2026-06-26T11:48:29Z")

</div>

> [@crackwitz](#):
>
> don’t do that, unless you mean to understand the standard model more deeply. my suggestion was a hypothetical for situations other than yours. the standard distortion model is fine.

It wasn’t a fisheye lens, so, yes, I think that the standard distortion model is fine for it. (There’s a limited number of lenses it could be, but I’m not sure which of them it was; that information was not explicitly recorded. And since it’s a c-mount, the camera couldn’t record that information automatically.) But I don’t know much about it, either. I need to know enough to know when/if I’ve recorded garbage.

> [@crackwitz](#):
>
> your issue is that the calibration data was gathered wrong, and perhaps the “experimental setup” might be flawed.

Agreed.

> [@crackwitz](#):
>
> that is also why stereo calibration functions support _using_ individual intrinsic calibration results during stereo calib.

Yes. But I think I’m going to use one script, while collecting them separately from the stereo frames, so that in saving the directory structure with images in it, I can document what constitutes good images for determining intrinsic parameters and stereo calibration all at once.

> [@crackwitz](#):
>
> > [@marisano](#):
> >
> > calibrateMultiview()
> 
> that’s a new function coming with v5.0. it _should_ be worth checking out. I have never used that before and don’t know its requirements.

I could also pair different of the four cameras together, to see if that helps. The biggest problem there is that I don’t have a good sense of how far some of the cameras were from one another. (They’re arranged in two stereo pairs, but I don’t know how “wide” the resultant rectangle was. I’ll assume it must have been similar to what it is today.) I can see that we’re going to need new data, but I already expected that. So, I think I’d better explore all the possibilities of the existing data to dismiss any doubt.

> [@crackwitz](#):
>
> first, resolution _defines_ what a pixel is. it is not expressed in pixels. I’ll assume you meant accuracy, or maybe reprojection error.

I mean the smallest size by which the visual system perceives visual information. Can one respond with greater accuracy and precision than they can actually visually perceive the world?

> [@crackwitz](#):
>
> if you go the “_stereo_ vision with disparity map” path, at some point the images get “rectified”

I may not be applying it entirely correctly yet, but I do already call `stereoRectify()`. However, since I’ve only used it with cameras of the same resolution, I never bothered checking `newImageSize`. Also, the images are so poorly rectified that the horizontal lines I draw along it never intersect the same part of both images, although reducing the rms usually gets them to align more closely.

> [@crackwitz](#):
>
> so either one image gets blown up, or the other’s “excess” resolution gets thrown away.

But either way, the accuracy of the calculated distance to a given point is not reduced, right?

One last thing: Great emphasis is placed on reducing the rms. However, from this discussion, I see that doing so it not always meaningful. So, under what circumstances should the rms _not_ be trusted? When the epipolar lines fail to be horizontal is one of them. Discordant focal lengths and distortion coefficients is another. Is there some value for which the rms is so low that these other issues can’t have arisen, or is it always wise to have a look at (and to understand) these other variables?