# Question about triangulation issues with OpenCV - gait analysis project

**URL:** <https://forum.opencv.org/t/question-about-triangulation-issues-with-opencv-gait-analysis-project/22050>\
**Category:** Uncategorized\
**Tags:** calib3d\
**Created:** [June 6, 2025, 2:12pm UTC](https://forum.opencv.org/t/question-about-triangulation-issues-with-opencv-gait-analysis-project/22050 "2025-06-06T14:12:20Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![NICCOLO\_VITO\_VACCA](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/niccolo_vito_vacca/32/12776_2.png) [@NICCOLO\_VITO\_VACCA](https://forum.opencv.org/u/NICCOLO_VITO_VACCA)\
**Post date:** [June 6, 2025, 2:12pm UTC](https://forum.opencv.org/t/question-about-triangulation-issues-with-opencv-gait-analysis-project/22050/1 "2025-06-06T14:12:21Z")

</div>

Hi everyone,

I’m working on a university project in collaboration with two companies, where we are developing a gait analysis program to evaluate patients’ walking patterns during rehabilitation in a hospital.

We are using Pose2Sim as the main framework, which internally uses **`cv2.triangulatePoints`** to compute 3D coordinates from 2D keypoints.  
Our setup consists of **4 cameras** , and the goal is to generate a `.trc` file with accurate 3D keypoints to extract gait parameters.

Here is our pipeline:

- We use **rtmlib** with **CPU** as device and **OpenVINO** for pose estimation.
- Intrinsic calibration works well, and pose estimation results look correct.
- Synchronization also appears to be working.
- However, when performing the triangulation step with **`cv2.triangulatePoints`** , the resulting 3D coordinates are not accurate. Keypoints appear unstable and imprecise in space.

We have tested this with:

- **2 and 4 camera setups**
- **Extrinsic calibration** performed both with _scene-based calibration_ and with _checkerboard calibration_.

Unfortunately, the same issue persists in all cases: the `.trc` coordinates remain inaccurate even when applying filtering.

We verified that the input 2D keypoints are correct and well synchronized, so we suspect the problem may be related to:

1. Possible errors in **extrinsic calibration** (improper relative camera transforms?).
2. Incorrect assumptions about the **projection matrices** passed to `cv2.triangulatePoints`.
3. Limitations of linear triangulation when using certain camera angles or baseline distances.

At this point I would like to ask:

- Are there known pitfalls when using **`cv2.triangulatePoints`** for gait analysis with multi-camera setups?
- Could inaccurate extrinsic calibration be the main cause in such cases?
- Are there better practices or recommendations when building projection matrices for this function?
- Should we consider more advanced triangulation methods (nonlinear optimization, bundle adjustment)?

Any suggestions or insights would be greatly appreciated!

Thank you very much in advance.