# Coordinates of touching the image projected from behind. (touchwall)

**URL:** <https://forum.opencv.org/t/coordinates-of-touching-the-image-projected-from-behind-touchwall/1584>\
**Category:** Uncategorized\
**Tags:** practical\
**Created:** [February 16, 2021, 12:29pm UTC](https://forum.opencv.org/t/coordinates-of-touching-the-image-projected-from-behind-touchwall/1584 "2021-02-16T12:29:27Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Laszlo\_Ivak](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/laszlo_ivak/32/1021_2.png) [@Laszlo\_Ivak](https://forum.opencv.org/u/Laszlo_Ivak)\
**Post date:** [February 16, 2021, 12:29pm UTC](https://forum.opencv.org/t/coordinates-of-touching-the-image-projected-from-behind-touchwall/1584/1 "2021-02-16T12:29:27Z")

</div>

As a rookie, I greet everyone on the forum! Help me please!

I would like advice on how I could implement openCV:

- Rear projected optical image on a 2m x 2m transparent screen.
- person touches the canvas with his hand
- 2 cameras above the plane of the screen in the xy (rectangular) position

– the coordinates of the touch point of the desired output  
– the required accuracy is 10-15cm

What do you suggest?  
Thanks for your help!

---

<div class="post-metadata">

**Author:** ![kbarni](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/kbarni/32/26_2.png) [@kbarni](https://forum.opencv.org/u/kbarni)\
**Post date:** [February 16, 2021, 1:10pm UTC](https://forum.opencv.org/t/coordinates-of-touching-the-image-projected-from-behind-touchwall/1584/2 "2021-02-16T13:10:45Z")

</div>

Maybe you have some example images?

---

<div class="post-metadata">

**Author:** ![Laszlo\_Ivak](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/laszlo_ivak/32/1021_2.png) [@Laszlo\_Ivak](https://forum.opencv.org/u/Laszlo_Ivak)\
**Post date:** [February 16, 2021, 2:07pm UTC](https://forum.opencv.org/t/coordinates-of-touching-the-image-projected-from-behind-touchwall/1584/3 "2021-02-16T14:07:04Z")

</div>

Thank you for your attention!  
I quickly drew one on the idea. 🙂

 ![touchwall1](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/c/c8102cbbc72f576470b17bf7d6077909c6533889.jpeg)

---

<div class="post-metadata">

**Author:** ![kbarni](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/kbarni/32/26_2.png) [@kbarni](https://forum.opencv.org/u/kbarni)\
**Post date:** [February 16, 2021, 2:32pm UTC](https://forum.opencv.org/t/coordinates-of-touching-the-image-projected-from-behind-touchwall/1584/4 "2021-02-16T14:32:24Z")

</div>

Use [triangulation](https://en.wikipedia.org/wiki/Triangulation_(surveying)).

Probably you have to fiddle with the formula, but basically you have to resolve the triangle CxCyT (CamX,CamY,Touch) where you know the angles, the position of the cameras and the distance between them.  
To get the relative angle of a point to the camera, you need to know the FOV (field of view angle) of the camera: `A=(2*P/W-1)*FOV` (P=point position; W=image width)

---

<div class="post-metadata">

**Author:** ![Laszlo\_Ivak](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/laszlo_ivak/32/1021_2.png) [@Laszlo\_Ivak](https://forum.opencv.org/u/Laszlo_Ivak)\
**Post date:** [February 16, 2021, 2:43pm UTC](https://forum.opencv.org/t/coordinates-of-touching-the-image-projected-from-behind-touchwall/1584/5 "2021-02-16T14:43:18Z")

</div>

Thanks Kbarney!  
I am a 57 year old automation engineer. The geometry part of the task is not a problem. I have no experience using OpenCV, I am looking for help because I need to solve this problem quickly.

---

<div class="post-metadata">

**Author:** ![kbarni](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/kbarni/32/26_2.png) [@kbarni](https://forum.opencv.org/u/kbarni)\
**Post date:** [February 16, 2021, 3:00pm UTC](https://forum.opencv.org/t/coordinates-of-touching-the-image-projected-from-behind-touchwall/1584/6 "2021-02-16T15:00:03Z")

</div>

As of the computer vision part: to make things easier, use a dark background.

Crop the part of the image that is close to the canvas (the white part of the camera view in your drawing). Threshold the image to get the pixels of the hand. Use the [Mu moments](https://docs.opencv.org/3.4/d8/d23/classcv_1_1Moments.html) to get the center of the hand:

```
cvtColor( src, gray, COLOR_BGR2GRAY );
gray_crop = gray[10:20,:,:] #crop lines 10-20
threshold( gray_crop, t, thresh_val,255,THRESH_BINARY ); #you need to measure thresh_val
Moments m = moments(t,true);
if m.m00<10: # less than 10 pixels active
    print("no hand detected")
else:
    hand_x = m.m01/m.m00
    print("hand coordinate: {hand_x}");

```

Probably you can refine the approach, but this is probably the fastest solution.

---

<div class="post-metadata">

**Author:** ![Laszlo\_Ivak](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/laszlo_ivak/32/1021_2.png) [@Laszlo\_Ivak](https://forum.opencv.org/u/Laszlo_Ivak)\
**Post date:** [February 16, 2021, 3:04pm UTC](https://forum.opencv.org/t/coordinates-of-touching-the-image-projected-from-behind-touchwall/1584/7 "2021-02-16T15:04:52Z")

</div>

Thanks!  
I’ll see, I’ll try.

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [February 16, 2021, 3:43pm UTC](https://forum.opencv.org/t/coordinates-of-touching-the-image-projected-from-behind-touchwall/1584/8 "2021-02-16T15:43:46Z")

</div>

if you want to register actual touch, that has been done with acrylic, lights into the glass from the edges, and a camera behind it to see the illuminated fingers.

you’ll find stuff via “FTIR” and “multitouch”

> **[Total internal reflection | Frustrated TIR](https://en.wikipedia.org/wiki/Total_internal_reflection#Frustrated_TIR)**
>
> If reflection is to be total, there must be no diversion of the evanescent wave. Suppose, for example, that electromagnetic waves incident from glass to air at a certain angle of incidence are subject to TIR. And suppose that we have a third medium whose refractive index is sufficiently high that, if the third medium were to replace the second (air), we would get a standard transmitted wavetrain for the same angle of incidence. Then, if the third medium is brought within a few wavelengths of the ...

going by your sketches, if you use cameras to look from the side, you’ll see fingers approaching the surface. that information, how far away a finger/hand is, can allow even more interesting effects.

I would make sure the camera’s optical axis lies precisely on the surface of the glass.

as kbarni demonstrated, you’d start by thresholding.

I would then find the first scanline with any white pixels (closest distance to glass), because that’s the tip of the nearest object.

 ![image](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/4/4bd43aea08972eb76511fee669d8ebd40bba9e11.jpeg)  
 ![image](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/7/7d51edcb4f07e6f3ee6d443035ae43f442a6bd64.png)

---

<div class="post-metadata">

**Author:** ![Laszlo\_Ivak](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/laszlo_ivak/32/1021_2.png) [@Laszlo\_Ivak](https://forum.opencv.org/u/Laszlo_Ivak)\
**Post date:** [February 17, 2021, 9:17am UTC](https://forum.opencv.org/t/coordinates-of-touching-the-image-projected-from-behind-touchwall/1584/9 "2021-02-17T09:17:52Z")

</div>

Hi Crackwitz!

Very useful help, thank you!