OpenCV 4.5.3; Visual Studio 2017 MFC; C++; Windows 10
Difficult to explain and too much code involved to put here but I will try to explain.
First of all this is a mult-thread app. In one mode, without the use of a camera, the main thread loops through a video file and “shows” the frames in the named window videoWindow.
In another mode, using a camera, the main thread calls a dialog box and from there the user selects the camera to use at which time the camera is initiated, a named window, cameraWindow, is created (actually a memory variable is set so that the main thread knows to create the cameraWindow) and then a thread is started and detached to stream the camera frames. Call this thread streamCamera.
Because there is a lot of processing that has to be done with the camera frames, before going into it’s loop, the streamCamera thread will start another thread and detach it (call this one the frameGrab thread) that will do the actual getting of the frames from the camera (3rd party camera software that will put the pixel data into a buffer which can then be read into Mat.data ) and then do some preliminary processing such as flip, thresholding, erode, dialate, etc and then put the original and thresholded frames in queues to be read by the streamCamera thread.
OK…so here is the fist question: The frameGrab thread that is detached has no highgui procedures in it whatsoever that I can see and yet it requires a waitKey(1) at the top of each loop or it won’t run. Why is a waitKey needed in a thread that has no highgui in it?
And here is the problem that started this post. If the user runs a video only in mode one with no camera, ie the cameraWindow is never created, the streamCamera and frameGrab threads are never started, etc. and then later tries to run in the camera mode, the waitKey(1) at the top of the frameGrab thread will fail and all processing the the frameGrab thread will cease and never get past the waitKey line…
I have recently ported over from OCV 2.4 to 4.5 and in that process made sure that all highgui (named windows, imshow, waitKey, setMouseCallback, etc) are all done in the main thread. So streamCamera at the most will set memory variables to have the main thread call a highgui procedure.
I tried to have the frameGrab thread also pass a memory variable to have the main thread call the waitKey but, at times when the video is getting set up, the main thread is not in a loop and therefor not able to catch the change in value of various memory variables, and inevitably I hit an area where, because of the lack of a waitKey the frameGrab thread will stop. Or, if I put a waitKey in frameGrab to account for the exceptions when the main thread is not in a loop, then that waitKey will crash and everything stops.
There has to be something that I am not taking into consideration. I have tried every possible workaround that I can think of and it still ends up crashing or just not functioning at some point.
Any ideas, no matter how out there, will be appreciated.