I have a project developed in c++ where I process images taken from the camera. Normally my project runs at 30fps without any problems. Now I am trying to add object detection algorithm to develop the project. However, the hardware I use can run the object detection model at around 10 fps. But 1 fps is enough for my project.
My problem is when I add object detection model to my project it works as 8-9 fps. Can I run object detection at 1 fps in the background and run normal operations at 30 fps? I tried to briefly explain what I want to do in the picture below. How can I do this?
I don’t need to detect for each frame, 1 in 30 is enough. But when I do it this way, the image waits until the detection process is finished in one frame. When the detection process is finished, the 29 frame works normally, then waits again. This is how I need to do it without waiting.
shove every 10th or 30th frame into a queue. the worker thread reads from the queue, runs inference, and puts the result into another queue, or whatever your program requires (post a GUI event?)
or “flip buffers”, always keep a reference to the most recent frame. let the worker thread grab a reference to it whenever it’s ready to process a new frame. that way it’ll run as fast as it can on the hardware, but if it runs slower than anticipated, frames don’t queue up.
I was having trouble running libtorch with thread. I just did as you said. After the thread completes the process, it takes a new image and continues to process it. The model is running slow as it runs in the normal main thread. But even if it works slow this way, it works for me.
