Starting a 3rd party program in an opencv window and controlling mouse clicks from my program

C++
1

I don’t know if this is possible but what I want to do is control the mouse clicks in a 3rd party program (ie arcade game) from within my program.

Would it be possible to, I don’t know, create a namedWindow, somehow start this 3rd party program within that window, and then be able to control the mouse clicks in the game since it is running in a namedWindow that my program created and controls?

Any ideas if this is possible?

Ed

2

that’s an os, not an opencv thing
(and you’ll have a hard time, finding something “portable”)

definitely impossible and maybe not even nessecary

3

berak,

As is my usual MO I usually have to ask a stupid question first to get my mind in gear to better see the problem. I “believe” that if I can define the ROI in the camera view of the area of the screen that contains the running program, I can then do a mouse click on the screen withing that ROI. If the mouse click is in an area that the arcade game sees as a hit, the arcade game should act according to its own programming.

In other words, I should be able to simulate a mouse click anywhere on the screen(s) regardless of where my main program window is and not just within the area of my own program. I’ve never done this because I have never had the need to try but that should work. I’ll have to play around with this. The only OCV thing here, then is the ROI of the Mat that is created from the camera stream. After that, I have no control of the 3rd party program of course, but I should still be able to produce mouse clicks in that area of the screen assuming a good homography calibration.

Thanks

Ed.

4

you only need the position / size of that window, not “physical access” to it
and you’ll need to transform coords from

camera -> screen -> game window