Hi, I’m new to opencv, we run a dairy farm and have an automatic teat spray which is just a couple of fixed spray nozzles in the race that the cows exiting the dairy walk over… currently the spray is triggered by a PE sensor which isn’t accurate, when the cow speed changes. Is it possible for opencv to track the cows position and speed and adjust the trigger position accordingly? Do you think it would be more or less reliable than a PE sensor. Where would I start? I just a Jetson nano with pi cameras, windows PC on hand and I’ve ordered an oak-d-Poe camera. Thanks
I’d aim to get rid of tracking and “trajectory prediction” and any kind of latency. or at least reduce, if convenient.
put the spray within the view of the camera. camera sees teat in the right place, camera triggers spray.
as for the triggering: yeah neural networks is the option to go with. there are a bunch of models, with weights (trained), that you can fine-tune to your application.
you’ll basically have to train the model to recognize the object class that you’re interested in: cow teats. you will have to collect images for training. those ideal come from the camera you’re planning to use, in the exact intended environment (everything in its position). no googling for pictures, no “phone pics”.
you can first record video from the camera, of plenty of cows walking by. more than dozens, hundreds I’d say. take that offline, into some annotation workflow (perhaps cvat? IDK), where you’ll run through the video and annotate lots of instances of teats. with practice, in a good annotation tool, that could take just a few seconds each.
given a model that can give you teat detection boxes, you can then define a region in the view that should contain teats. when the model gives you detections in that region, that’s the trigger. to gain performance, you could crop the view towards the region, so the model doesn’t have to analyze as much of each frame.
since you get detections (or not) for every frame, you’ll have to have some hysteresis logic: when there are (enough?) teats for 1/several consecutive frames, throw the trigger, and remain “hot”. when there are no (or not enough?) teats for 1/several consecutive frames, reset back to “cold”. that also helps the electromechanics for the spray not go on and off like mad.