Plate OCR from RTSP cameras at night: what actually helped you most?

I work on a plate and text reading tool for CCTV cameras (it is called VisuDock Live OCR, so full disclosure, it is my product). Night time is where most of the problems show up, and I would like to hear what other people found works.

The things I keep running into:

  • infrared light bouncing off the plate and turning it into a white box
    • motion blur when the camera drops to a slow shutter
    • plates that are only 40 to 60 pixels wide in the stream
    • headlights washing out the picture
      For people who have built their own pipeline: what made the biggest difference for you? Camera side (shutter, IR level, white light, angle) or software side (cropping, deblurring, using several frames and voting on the result, super resolution)?

And did you keep the main stream, or did you find the sub stream was good enough for reading?

Any numbers would help (plate width in pixels, what read rate you got). Thanks.

Thanks, this is very useful. The point about the white box being IR bouncing off the retroreflective film, and fixing it at the camera before touching software, is the part many people skip.

Two follow-up questions if you have time:

  1. When you majority-vote across frames, do you vote per character position, or on the whole plate string? I can see per-character failing when the plate length changes between frames.
    1. How many sharp frames do you usually need before you trust a result? Is there a rule you use, like a minimum plate width plus a minimum agreement count?
      Thanks again for the detail.

vote on the whole string, not per character. per character can stitch together a plate that never appeared in any single frame, especially when the length changes. 3 identical reads out of 5 sharp frames is a workable cutoff, below that just keep collecting.

One more experiment for the white-box problem: try a polarizer rated for the illuminator’s IR wavelength, with polarizing film on the light and a rotatable filter on the lens. Compare plate crops with and without it at the same exposure. Retroreflective return may not respond like ordinary glare, and the filter also costs light, so it may not help this setup. Keep the shutter limit suggested above fixed for the comparison, and check whether the characters are actually less clipped and still sharp on moving vehicles.

Thanks both, this is helpful. Rachel, voting on the whole string makes sense, and “3 identical reads out of 5 sharp frames” is a simple rule that is easy to explain to users.

Charles, good idea on the polarizer. One practical note: a polarizing filter usually costs one to two stops of light, so with the shutter fixed you would need to raise the illuminator level or gain to compare fairly. Retroreflective plate film may also scramble polarization, so the gain could be smaller than with ordinary glare. Worth testing on one camera before buying for all of them.

A question for anyone who has tried it: for the evidence photo of each vehicle, do you keep the sharpest or widest crop, or simply the last frame that matched?