Extract timestamp of each frame in a video - CAP_PROP_POS_MSEC incorrect timing issues

Python
1

I have captured several videos from a USB camera connected to NVidia Jetson TX2 board using openCV VideoCapture and VideoWriter methods. The video acqusition is triggered by digital I/O in the NVidia board and I am recording start and end timestamps of when video acqusiiton was triggered and stopped. (see code below) The video acquisiton is triggered by an external device connected to the NVidia board.

# Setup IO
GPIO.setmode(GPIO.BCM)
GPIO.setup(26,GPIO.IN) # set pin 26 as digital IN - start video capture when this goes LOW

vidCount = 0
start_timestamp = False
end_timestamp = False
acq_trigger_started = False
acq_trigger_ended = False
acq_trigger_wait = False

columns = ['Video_Count', 'Layer', 'Speed', 'Start_Timestamp', 'End_Timestamp', 'Time_Diff']
lst = []
vfc = 1
vR = 10
layer = 3

try:
        
    cap = cv2.VideoCapture("/dev/video1")

    # check if camera opened successfully
    if (cap.isOpened() == False):
        print("Error initializing camera stream, check camera connections. Exiting ...")
        exit(0)
    
    while (cap.isOpened()):

        if not acq_trigger_wait:
            print("Waiting for start video acqusition trigger . . . ")
            acq_trigger_wait = True

        while(GPIO.input(26)==0):
       
            if not acq_trigger_started:

                
                fps = cap.get(cv2.CAP_PROP_FPS)
                print('Video frame rate={0}'.format(fps))
                
                start_time = datetime.datetime.now() # record start_timestamp here

                frame_width = int(cap.get(3))
                frame_height = int(cap.get(4))

                size = (frame_width, frame_height)                
                
                vidCount = vidCount + 1
                vidName = "vid" + str(vidCount)
                result = cv2.VideoWriter(str(vidName) + ".avi",cv2.VideoWriter_fourcc(*'XVID'),30, size)

                print('The video acqusition for vid{0} has started at {1}'.format(vidCount,start_time))
                acq_trigger_started = True
                acq_trigger_wait = False
                acq_trigger_ended = False
                                

            frame_exists, frame = cap.read()
            #print('frame count: {0}'.format(frameCount))
            #print('ret: {0}'.format(ret))

            if frame_exists:

                    # Write the frame into the
                    # file 'filename.avi'
                    result.write(frame)

                    # show frame
                    cv2.imshow('Live Video', frame)
                    
            else:
                print("Can't retrieve frame - stream may have ended. Exiting..")
                break
        
        if not acq_trigger_ended:
            if vidCount >= 1:
                end_time = datetime.datetime.now()
                print('The video acqusition for vid{0} has ended at {1}'.format(vidCount,end_time))
                acq_trigger_ended = True
                acq_trigger_started = False 
                
                # Closes all the frames
                result.release()
                cv2.destroyAllWindows()
                print("The video was successfully saved")
                
                time_diff = (end_time - start_time)
                execution_time = time_diff.total_seconds() * 1000
                lst.append([vidCount,layer,vR,start_time,end_time,time_diff])
                
                # Update layer count and speed
                vR = vR+10
                
                if(vR > 50):
                    vR = 10
                    layer = layer + 1

except KeyboardInterrupt:
    # user pressed ctrl + C
    print("Program terminated by user. Exiting gracefully . . . ")
    if(cap.isOpened()):
        cap.release()
        result.release()
    GPIO.cleanup()
    cv2.destroyAllWindows()

    # Save pandas dataframe to excel/csv
    video_timestamps = pd.DataFrame(lst,columns=columns)
    video_timestamps.to_excel('Video_Timestamps_' + vfc + '.xlsx')
    exit(0)

At the end, a file Video_Timestamps_1.xlsx containing start and end timestamps for each video is saved. I am reading this file using pandas in a different script and the snapshot of this data looks as follows:

# open and read file containing start and end timestamps of the videos
df_vidTimes = pd.read_excel(SRC_FOLDER + "Video_Timestamps_1.xlsx")

Video_Timestamps

Next, I want to get timestamps of each frame in a video. For this, I am using cv2.CAP_PROP_POS_MSEC property that calculates position of each frame in the video based on frame rate cv2.CAP_PROP_FPS. However, there seems to be a big mismatch in “real” timing of the video (recorded in excel) from that calculated by adding frame times using cv2.CAP_PROP_POS_MSEC.

For example, lets consider video-1 that has a start_timestamp of 2021-09-24 19:43:35.167, end_timestamp of 2021-09-24 19:43:51.088 which results in a video of length 00:00:15.921000. For the same video, using cv2.CAP_PROP_FRAME_COUNT property results in a value of 208 (208 frames in this video). Doing some simple calculations based on “real” timestamps, given the video length is 15.921 seconds and I had specified frame rate of 30 in VideoWriter object: result = cv2.VideoWriter(str(vidName) + ".avi",cv2.VideoWriter_fourcc(*'XVID'),**30**, size), shouldn’t I expect 15.921*30 = 477 frames instead of 208?

calculating total length of the video using cv2.CAP_PROP_FPS and cv2.CAP_FRAME_COUNT results in time = float(frame_count)/fps i.e. 208/30 = 6.933 seconds which is vastly different from 15.921seconds!

  1. Why is there is discrepancy in total number of frames in the video and total length of the video?
  2. Does VideoWriter method guarantee specified fps?
  3. What is the correct way to get timestamps of each frame of the video?
2

related: