Access webcam from colab

Python
, ,
1

I built a Lane Detection using U-Net Architecture which works perfectly fine.

I create a “run” function which does this:

  • Take an image as input
  • Resize the input to match the model size
  • Call model.predict()
  • Extract R,G,B channels from prediction
  • Rescale to the original image size
  • Draw on the input image Return the result
def run(input_image):
    h,w,d = input_image.shape
    network_image = input_image.copy()
    network_image = cv2.resize(network_image, (160,80), interpolation=cv2.INTER_AREA)
    network_image = network_image[None,:,:,:]
    prediction = model.predict(network_image)[0]*255
    R,G,B = rgb_channel(prediction)
    blank = np.zeros_like(R).astype(np.uint8)
    lane_image = np.dstack((R,blank, B))
    lane_image = cv2.resize(lane_image, (w,h))
    result = cv2.addWeighted(input_image, 1, lane_image.astype(np.uint8), 1, 0)
    return result

So when I need to predict it on a test image, I do this:

f, (ax1) = plt.subplots(figsize=(20,10))
ax1.imshow(run(predict_image))
type or paste code here

Then it returns me the image with the lane being detected by the model:

image
image1320×868 181 KB

Now if I need to run it on a video:

from moviepy.editor import VideoFileClip
video_file = "/content/drive/MyDrive/Image Segmentation/videos/video_input.mov"
clip = VideoFileClip(video_file)
white_clip = clip.fl_image(run)
%time white_clip.write_videofile("/content/drive/MyDrive/Image Segmentation/Output/video_output.mp4",audio=False)

Now I want to run the lane detector in real-time using my webcam. How can I do this?

thanks!

2

what are you asking? are you asking how to use cv.imshow? how to use cv.VideoCapture?

3

I’m new to openCV so whichever will work in real-time when using my webcam.

That is, my webcam will point at a road then we can call the “run” function so that we see the lane being detected in real-time.

4

https://docs.opencv.org/master/dd/d43/tutorial_py_video_display.html

1 Like
5

I get this on colab:

image
image918×656 29.5 KB

6

obviously colab runs on some server on the internet. it can’t access your local webcam and it can only show you single images, not a video.

and why is that exit() call not working properly? it’s supposed to end the program right there in line 6, not enter the loop.

1 Like
7

Isn’t there any way colab can access my webcam?

I used the code in the link you sent me.

8

I found this link here which may help: Google Colaboratory

Can you suggest me how to out in my “run” function?

9

interesting code. I just tried it. it works… but I get around one frame per second. that’s because the code has to copy every frame, uncompressed, base64-encoded, as json, on demand, across the network.

a better idea would be to set up python and opencv locally. install python (version 3), run pip install opencv-python (or pip3 on linux), same install command with matplotlib (or use opencv’s imshow)

1 Like
10

But that’s the thing. I need to run my model on colab gpu as it taked too long to train the model locally.

11

you are mixing things up.

“running a model” is understood to mean “inference”, i.e. forward passes.

training uses a lot more computation than inference.

you train your model on colab using training data you’ve collected and labeled.

I believe your model was already trained and you just run it.

you can run your model on colab over the internet (from inside a moving vehicle using cellular internet with comparably high latency??), and accept the slow frame rate achieved by that piece of source code that transmit client-side webcam pictures to colab.

or you can run the network locally, which is feasible if it’s a small network (some can run on mobile phones) or your computer has a GPU with sufficient computational power.

1 Like