AR rock climbing wall

Python
1

Howdy folks.

I have a strong strategic interest in building something like this;

I know Python and basic cpp and js, and I understand the basics of opencv. Any hints?

1 Like
2

human pose estimation.

thatā€™ll give you points for hands and feet. match that to previously known (manually noted down) positions of these holds. if hand/foot is close enough to some hold, ā€œactivateā€ the hold.

add some perspective transforms to map between the walls and the camera view.

1 Like
3

Have you seen any solution with ā€œmultipleā€ perspective transforms?

Meaning, assume a wall of polygons with handholds and the distance from the camera is the z axis. Some handholds are further from the camera, others are closer to the camera. Can I find x y z coordinates with any degree of accuracy? How about coords for vertices of the polygons? Hands and feet?

If itā€™s unmanageable without additional hardware (beyond a webcam) Iā€˜ll look into it.

5

you can overengineer this, but thatā€™s a waste of time.

the ā€œmost expensiveā€ part of this is modeling the wall and holds.

make that cheap to do, so it takes a mere minute to model a wall. make that manual, because automating it is a waste of time.

make sure the camera and projector stay in their places for the duration of a game. I think thatā€™s not too much to ask for.

take a camera picture.

  • manually annotate the holds.
  • manually draw a quad (its four corners) that roughly represents the corners of the assumed-flat wall.

show a quad on the projector, such that you also see it on your screen. adjust its corners interactively, to coincide with the corners you picked previously.

now you have enough data.

holds need no specific detection. they donā€™t move. they are fixed regions.

the only thing that does need detection is the people.

in the video you linked, the registration/pose estimation is glitchy. people who ā€œdoā€ this, rather than ā€œoverengineerā€ this, usually get slightly glitchy results, but nobody cares, because it works well enough. so, donā€™t overthink it. their glitching is due to them running wall-camera pose estimation needlessly at runtime. they should do that once at the beginning, and keep that pose info.