latency is to be expected if you can’t process as quickly as the camera makes frames. it makes them at its own pace. you can’t throttle it. if you try, frames will queue up.
you must read every frame. if you don’t have the time to process it, discard it.
obviously your camera produces 17-18 fps. that is not in question. the processing you do is in question.