I’m using OpenCV in Python to read frames from an RTSP stream. I’m actually trying to read many streams for a multi-camera tool, but I have this problem with a single stream as well. So long as I call read() at least as fast as the video stream’s framerate, things work out. But if I fall behind and don’t call read() fast enough, then calls to read() hang for very long periods of time, like half a minute. It is effectively dead. There is no timeout option for read() or for configuring the VideoCapture object that I am aware of, and read() simply won’t tolerate a failure to maintain pace. I’m already running the frame retrieval on a background thread (bearing in mind that threading in Python doesn’t really amount to much, GIL), but ultimately, any solution that boils down “really really never ever miss a single frame or else it’ll hang” isn’t a sufficient solution. Any reasonable expectation would be that an app is permitted to request a frame from the stream whenever it is able to do so and receive the current or most recent frame without risking having VideoCapture veritably freeze up.
Does anyone have a solution in mind? I’m desperate to get this working and have no ideas what to try next. I’ve been hammering this thing for days and I’ve made basically no progress on this issue.
Thank you.
the Video I/O API is not designed to handle that.
run your VideoCapture in threads. read all frames. use those you need.
OpenCV releases the interpreter lock, so yes, python threads will work.
that’s the only solution you have.
or use respective media APIs directly.
I already said I was running it in a thread. It definitely doesn’t work if I can’t keep up and read every single frame without fail. Why does it crash hard if you can’t call read fast enough? Even if it can’t deliver a frame during such an occurrence, why can’t it return safely instead of locking up? All sorts of things could cause a computer to occasionally hiccup and fall behind momentarily. Is there really no provision to keep the application alive when that inevitably, occasionally happens?
Okay, thanks for the feedback. Do you understand why it works that way? Even if OpenCV has no choice but to handle a fall-behind erroneously instead of merely delivering the currently available frame (as one would expect) why must it freeze up for 20, 40, even 60 seconds? Why can’t it just quickly return a NULL or something seemingly sensible like that?
if it actually crashes, you can file a bug. that shouldn’t happen. you gave no details that would allow pinning the issue down.
it works that way because that’s the simplest implementation… and nobody has bothered to improve it.
improvements would involve changes in the API, and then in the backends. feel free to discuss that on the github in an appropriate issue (search and you might find related/old ones). that’s where most of the developers (of the library) are.
No, that was the wrong term, a generalization. It doesn’t crash. It almost freezes, and it’s hard to be confident that it would operate even in this problematic capacity for long stretches of time, but perhaps it would. Note that it doesn’t steadily slow down. It hums along, delivering frames in fractions of a second – obviously crucial for video – and then suddenly a call to read() simply blocks for, as I said, a minute.
I’m not sure what other information I would could provide from the API, function-calling level of usage that would help pin it down. I presume anyone deeply familiar with the internal workings of read() would have little trouble from my description finding where in the code read() suffers an opportunity to block on what should sensibly be a near-real-time call. In fact, as I said, it seems to me that one solution is to not even attempt to solve the underlying problem, allow read() to fail to deliver a frame, but then offer a timeout option that forces read() to abandon the attempt after, oh say, a tenth of a second, and return a NULL frame. That would just end-run all the potential difficulty in actually improving the failure to capture a frame. Just give up quickly. That seems reasonable to me.
Thanks.
oh nevermind, you talk about RTSP streams. that’s OS-independent.