VideoWriter ouputs artifacts in some occasions

Python
1

Hello everybody!

I’m facing a strange issue while writing a set of frames as a video to disk by using the cv2.VideoWriter. In some occasions the output is rendered like this image:

render
render1287×1283 246 KB

This is basically the code:

out = cv2.VideoWriter(video_path, fourcc, 25, (420, 420))
for i in range(len(frames)):
    # Convert RGB to BGR, OpenCV expects BGR
    frame = frames[i][...,::-1] * 255
    frame = cv2.resize(env_frame, (420, 420), interpolation=cv2.INTER_AREA)
    out.write(frame)
out.release()

The frames are sourced from different Reinforcement Learning environments like CartPole, Minigrid or Obstacle Tower. I’ve got no issues if I record videos based on CartPole. Initially I had issues with Minigrid, but resolved these by grabbing freshly rendered frames. I did not get it to work for Obstacle Tower.

Because of Minigrid I figured that using references might be the issue as the frames may also be used elsewhere. So I used deepcopy for retrieving the frames. However, that did not resolve my problem for Obstacle Tower.

I ensured that the frames feature np.int32 values in the range 0 to 255.

I’m not sure if this is somewhat a scaling or aspect ratio issue. The Obstacle Tower frames’ dimensions are 84x84. Minigrid and Cartpole exceed my target video resolution of 420x420. Well, I can decrease Minigrid to 84x84 as well and get a good result. It remains unclear to me why this issue persists on the other source.

Does anybody have an idea on how to debug this problem? What other information could I provide?

edit: I tried varius codecs like MP4V, DIVX or MJPG.
edit2: I also plotted each image before calling write() to verify that they appear like expected.

2

when you say “plot”, does that mean you check frame's contents right before out.write, not earlier?

that picture you show looks like badly formatted channel data, perhaps some transposition or unexpected number of channels.

make sure your frame has 3 color channels. also make sure that you pass uint8 data in the range of 0…255, or float data in the range 0…1.

try passing frame.copy() to out.write. that shouldn’t make a difference, but it may. the copy will be contiguous memory, no weird strides (::-1 can cause that).

1 Like
3

Converting the array to np.uint8 was the solution! Thanks!

I also created a tiny archive (10mb) to reproduce this issue:
https://drive.google.com/file/d/1B9OSUgFyvUMZYXRBJaMMzd4AJdexj-4S/view?usp=sharing