How to Blend / Add / Insert an Image to Video and Real Time Live Stream via Webcam to Tensorflow bbox coordinates to spawn my image around that rectangle with OpenCV Python

Python
,
1

This is my first topic in the forum and there is a subject I struggled a lot while doing lots of testing to apply the feature I want with opencv in python.

# This gives the coordinates of the detected objects' bbox coordinates what is in my for loop
y1 = (int(box[i,0]*height))
x1 = (int(box[i,1]*width))
y2 = (int(box[i,2]*height))
x2 = (int(box[i,3]*width))


# After that I say here "if the detected object is a car and its ID is 1, then blend the image on the webcam frame

if classes[0][i] == 3 and i == 1:

                        (x_center,y_center) = (x2 + x1)/2, (y2 + y1)/2

                        added_image_2 = cv2.addWeighted(frame[: , x1:y1, :],alpha,img[:,75:75,:],1-alpha,0) # img[0:75,0:75,:] img[0:100,0:100,:]

                        frame[y1:y2+75 , x1:x2+75] = added_image_2

However, I always get this error:

added_image_2 = cv2.addWeighted(frame[: , x1:y1, :],alpha,img[:,75:75,:],1-alpha,0) # img[0:75,0:75,:] img[0:100,0:100,:]
cv2.error: OpenCV(4.4.0) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-q0nmoxxv\opencv\modules\core\src\arithm.cpp:669: error: (-209:Sizes of input arguments do not match) The operation is neither ‘array op array’ (where arrays have the same size and the same number of channels), nor ‘array op scalar’, nor ‘scalar op array’ in function ‘cv::arithm_op’

Before this, I applied this feature I want with mouse coordinate successfully with this line of codes:
# Select the region in the background where we want to add the image and add the images using cv2.addWeighted()
added_image = cv2.addWeighted(frame[mouseY:mouseY+img_height , mouseX:mouseX+img_width,:],alpha,img[0:100,0:100,:],1-alpha,0) # img[0:75,0:75,:]

     # Change the region with the result
     frame[mouseY:mouseY+img_height , mouseX:mouseX+img_width] = added_image

and this is the result of it:

1084×610 111 KB

What I want this to insert / add this image on where the detected object for ID is 1

1084×610 160 KB

I have taken to apply a different concept, thanks to this tutorial example from here:
add-image-to-a-live-camera-feed-using-opencv-python/

If anyone would guide me how to solve this issue I will be very glad, thank you very much.

2

I don’t really understand what do you want to achieve, but the coordianted are badly messed up in your addWeighted function.

Keep it simple:

bHeight = y2-y1
bWidth = x2-x1
frame[y1:y2,x1:x2,:] = frame[y1:y2,x1:x2,:]*alpha + image[0:bHeight,0:bWidth,:]*(1-alpha)

If needed, resize the imageto match the size of the box you want to replace.

1 Like
3

I think he tries to add an image for the detected objects, like where in the tv series named “Person of Interest”

target image - person of interest - samaritan

another image target logo for the detected object

It will be really cool if we would achieve such feature in opencv python too that I have seen same example in java and C++; but not with opencv Python.

@kbarni , what do you think about how would we add such feature? thank you.

4

@kbarni I try to add an image via overlay image on the detected object’s coordinate to make everything visually more attracting and cool which I didn’t want to use rectangles all the time.

I tried your fixed code and it works but now I have another problem.

When my overlay image on the detected target goes the boundary of the right line and bottom line of the webcam frame, then I get this size error:

Properties-of-Rectangle
Properties-of-Rectangle750×273 28.8 KB

cv2.error: OpenCV(4.4.0) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-q0nmoxxv\opencv\modules\core\src\arithm.cpp:669: error: (-209:Sizes of input arguments do not match) The operation is neither ‘array op array’ (where arrays have the same size and the same number of channels), nor ‘array op scalar’, nor ‘scalar op array’ in function ‘cv::arithm_op’

Do you know how to prevent and solve this size problem when my overlay image’s boundary tries to go beyond of the webcam frame where the overlay image spawns for the detected object near the right and bottom line of the webcam frame? thank you.

I will be very glad if you would guide me about how to apply such feature.

5

Anyone know how to prevent to get this error that I have mentioned at my above comment which when I my overlay image on the detect object moves close to BC and AB line then I get this size problem that says " sizes are not same "? thank you.

cv2.error: OpenCV(4.4.0) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-q0nmoxxv\opencv\modules\core\src\arithm.cpp:669: error: (-209:Sizes of input arguments do not match) The operation is neither ‘array op array’ (where arrays have the same size and the same number of channels), nor ‘array op scalar’, nor ‘scalar op array’ in function ‘cv::arithm_op’

So, is it possible to update the size of the overlay image simultaneously with resize function according to rectangle’s changing shape on the target’s object detection in while loop? thank you.

6

I am not much of a Python OpenCV user, but as far as I can tell, you are exceeding the size of destination image. Easiest way to solve this problem would be clamp the range of copy:

# Clamp copy range
width, height = cv.GetSize(frame)
y2 = min(y2, height)
x2 = min(x2, width)

# Copy
bHeight = y2-y1
bWidth = x2-x1
frame[y1:y2,x1:x2,:] = frame[y1:y2,x1:x2,:]*alpha + image[0:bHeight,0:bWidth,:]*(1-alpha)