(SOLVED) Need help with LIVE video calibration

Python
1

I’m very new to OpenCV but learn best from doing and thought there would be a lot more support for learning and tutorials but it seems what I can find is in C++ or its simply so old it doesn’t seem to apply anymore.

My goal is to calibrate a USB camera mounted in place so far I have been able to get an YAML file of the Matrix and Coefficient (Realizing this may not be enough to remove fisheye) I started looking at other tutorials including the one in the documentation it seems to have a plethora of info on using it for removing fisheye using Still images BUT I would like to have a webcam running for machine vision without fisheye so I can accurately account for real world measurements. The problem is I cant seem to find a python tutorial on how to do this without using still photos.

The ideal way I wanted to do this is take the VideoCapture frame by frame and remove the fisheye from each frame before outputting them to the cv.imshow(“Undistorted Webcam”, frame) I would post some of my attempts at this but quite frankly I have scrapped so much code so far as I’m lost.

TL;DR Is it even possible to remove distortion/fisheye from a live stream from a webcam. If so is there any documentation or anywhere I can learn more about this. If not any info you can provide would help greatly.

2

great ! have a look at the undistortion part here (at the bottom)

all you have to do for live images now is :
(once, at start of your program)

then, for each img from the livestream, call:
cv.remap

gl !

3

The github link you provided seems to be broke as it links directly to the github.com I appreciate the timely reply!

4

sorry link corrected to OpenCV: Camera Calibration

5

Ah yes this is the exact documentation I was following the last few steps you referenced did trip me up as following it in order I tried to attempt to run all of it in the order listed on the site little to say I did learn thats not the correct way will update you as I try your suggestion of remap! Much love.

6

Ok so to add to my intuition since you seem very helpful this is the tutorial I followed as I wanted to attempt the circle grid, I used this code to generate the YAML file and it does not clearly explain how to use the file to populate the data from the file to be used as you mentioned above at the start of the program is that code block referenced at the bottom the only thing needed or is there more to be done to use this?

7

so, problem now is, how to get the cam_mat and dist_coefs from the yaml back into your program ?

8

Exactly as I have no experience with arrays in this fashion of importing a yaml file

I’m sure its stupidly simple I’m probably just over thinking it possibly.

1 Like
9

import numpy as np
import cv2 as cv
import yaml

with open(‘calibration1.yaml’) as f:
loadeddict = yaml.full_load(f)
mtxloaded = loadeddict.get(‘camera_matrix’)
distloaded = loadeddict.get(‘dist_coeff’)

cap = cv.VideoCapture(0)

while cap.isOpened():
ret, frame = cap.read()
#cv.imshow(‘Webcam Distorted’, frame)
print(“we are reading frames”)

#Grab Frames and attempt to undistort
h, w = frame.shape[:2]
newcameramtx, roi = cv.getOptimalNewCameraMatrix(mtxloaded, distloaded, (w, h), 1, (w, h))

dst = cv.undistort(frame, mtxloaded, distloaded, None, newcameramtx)

# undistort
dst = cv.undistort(frame, mtxloaded, distloaded, None, newcameramtx)
print("Undistorting")
    
# crop the image
x, y, w, h = roi
print("Cropped the image")

dst = dst[y:y+h, x:x+w]
cv.imshow('UNDISTORTED', dst)

#End loop when hit d
if cv.waitKey(20) & 0xFF==ord('d'):
    break

cap.release()
cv.destroyAllWindows()

10

wait, you don’t need to calibrate again, you already done that ! just use the yaml data !

cap = cv.VideoCapture(0)

while cap.isOpened():
    ret, frame = cap.read()
    #cv.imshow('Webcam Distorted', frame)
    print("we are reading frames")
    
    #Grab Frames and attempt to undistort
    h, w = frame.shape[:2]
    newcameramtx, roi = cv.getOptimalNewCameraMatrix(mtxloaded, distloaded, (w, h), 1, (w, h))
    # undistort
    dst = cv.undistort(frame, mtxloaded, distloaded, None, newcameramtx)
    print("Undistorting")
        
    # crop the image
    x, y, w, h = roi
    print("Cropped the image")

    dst = dst[y:y+h, x:x+w]
    cv.imshow('UNDISTORTED', dst)

    #End loop when hit d
    if cv.waitKey(20) & 0xFF==ord('d'):
        break
11

:man_facepalming: I tried to fix that before you noticed I just went to edit my post and it replaced the whole message lmao

12

The weird error im getting is this currently googled it but not exactly sure why its happening.

cv2.error: OpenCV(4.6.0) :-1: error: (-5:Bad argument) in function 'getOptimalNewCameraMatrix'
> Overload resolution failed:
>  - cameraMatrix is not a numpy array, neither a scalar
>  - Expected Ptr<cv::UMat> for argument 'cameraMatrix'
13

Incase you’re curious about whats in my yaml file here it is:

camera_matrix:
- - 477.94288469324334
  - 0.0
  - 348.2357848325608
- - 0.0
  - 478.68358318081016
  - 198.91483354517516
- - 0.0
  - 0.0
  - 1.0
dist_coeff:
- - -0.3769228917448197
  - 0.11101170472409036
  - 0.0045273638941951315
  - -0.0075160770990671906
  - 0.01006734907539387
14

and, what is the type(cameraMatrix) ?

theyre converting the data to simple lists , so you probably need to make it a numpy array again

15

I love you so much LMAO god ive been dealing with youtube, google and brute force for the past 12 hours.

1 Like