Is it possible to set batch size other than one with yolo models?

Python
1

I am loading a yolo model with opencv in python with cv2.dnn_DetectionModel(cfg,weights) and then calling net.detect(img) . I think I can get a speed-up per image using batches, but I don’t see any support for batch size other than one.

Is it possible to set the batch size?

1 Like
2

what is img, is it a blob… from an image?

how about blob from imageS, plural?

https://docs.opencv.org/master/d6/d0f/group__dnn.html#ga0b7b7c3c530b747ef738178835e1e70f

3

img is a (h,w,3) uint8 numpy array. No amount of fiddling with stacking, concatenating, putting in lists, etc, allowed me to pass more than one image into net.detect.

4

One thing that does, work:

net = cv2.dnn.readNetFromDarknet(cfg,weights)
net.setInputNames(["input"])
net.setInputShape("input",(batch_size,3,h,w))
blob = cv2.dnn.blobFromImages(image_list,*args,**kwargs)
net.setInput(blob)
net.forward()

And then you have to do your own non max suppression.

5

yep it does not accept batches. ;(
(same problem as in the “low level” sample )

unfortunately, it’s not as simple as calling blobFromImages() alone,
the whole post-processing (result parsing, NMS) has to be multiplied

one thing you might want to try is profile your batch code there against multiple forward() calls with a single images (testing all backends/targets you have)

6

I did profile it. The backends I have available to me are opencv and openvino, and the targets are cpu, intel integrated gpu 32 bits, and intel integrated gpu 16 bits. On cpu, openvino is faster and batch size doesn’t matter. On gpu, to my knowledge, openvino is required. 16 bits is faster, and throughput is significantly faster with batch size > 1. For example, the throughput on the images and model I am using is 70 ms/image at batch size 1 (on intel gpu with 16 bit inference) and is 50 ms/image at batch size 3.

1 Like