OpenPose 3D reconstruction from saved RGB stereoscopic video

I want to display several (x,y,z) keypoints of several body parts in OpenPose by CMU from a 3D reconstruction of saved, inputted video frames from a synchronized left and right camera. All of the tutorials I have seen seem to only work with developing a 3D reconstruction in real-time & from connected FLIR cameras, which I do not want to do; I would like to use my own videos/image frames. Is this possible in OpenPose, or does it only support a real-time feed from 2+ cameras?

the openpose network(s) you can use in opencv’s dnn run on single images, you don’t need a stereo pair at all.

also they compute only 2d (x,y) landmarks, not 3d points

have a look at the example code here

you probably came across this, but i’m afraid, we can’t help much with it.

Yes, I am aware of this, however I wish to utilize stereo pairs to get the z position of someone’s hand, therefore I would need (x,y,z) points for this. I believe a 3d reconstruction from stereo pairs/a disparity map would help achieve this.

Apologies, I just saw the edit you made with the extra link - yes, that is exactly what I am trying to reproduce, but not in real-time

i did not dive into the code yet, but they seem to expect a stereo calibration. did you do this before recording your stereo sequences ?

Yes, both cameras are calibrated, and I have the intrinsic parameters of both cameras.

1 Like

there is no 3d network there, right ?
z is triangulated from 2 2d openpose networks (running on each stereo image pair)

in other words, maybe it could also be done in opencv !

1 Like

Huh, I actually hadn’t considered that, I will have to look into how I can triangulate the position of z for each possible point in space & from each stereo pair in OpenCV; are there any starter code that I can look at that you can suggest?

have not found yet, how/where the triangulation is called from …

btw, i find stuff from mono images even more interesting :slight_smile:


Okay I will take a look through the repo!