Reproduce 2 successive perspectives transformations with a dot product

Python
1

Hello,

I have 3 photos taken from the same point of view at years 2014, 2017 and 2020. I’d like to align theses photos using controls points and a perspective transformation.

I’ve computed the following perspectives matrices :
M1 transform from 2017 to 2014
M2 transform from 2020 to 2014.

The alignment is very good !

I thought I could align the picture from 2017 to the picture from 2020 by combining 2 successive transformations (M1 following inverse of M2). It work great if I decompose the steps with 2 calls of warpPerspective()

M2 = np.linalg.inv(M2)
im = cv2.warpPerspective(im, M1, (w, h))
im = cv2.warpPerspective(im, M2, (w, h))

But if I try to combine the 2 matrices with a dot product, then the alignment is not as perfect.

M = np.dot(M1, M2)
im = cv2.warpPerspective(im, M, (w, h))

alignement
alignement1599×594 291 KB

Left is the picture from 2020, right the result of the 2 successive transformations and then on right comparison, the result of the dot product

I thought I’d get the same result. Anyone can help me to understand what’s can be wrong in this assumption ?

thanks you very much

2

Wrong order.

Linear algebra:

v' = M2 (M1 v)
   = (M2 M1) v

why did you reassign M2 to be inv(M2)? that is confusing. I’ll ignore that and expect that your issue is due to the wrong order of matrix multiplications.

3

Thank you, changing the order of the dot product solve the issue.

m2i = np.linalg.inv(m2)
m = np.dot(m2i, m1)

My apologies, it was a very simple mistake !

4

mind the “frames”, i.e. coordinate/reference frames, origins, that stuff. and mind the way each matrix transforms.

matrix mul is not commutative. if you have a common frame A and other frames B and C, and you have T_A_B (transforming from B into A), and T_A_C (from C into A), and you want to go from B to C, then you need T_B_A @ T_A_C = inv(T_A_B) @ T_A_C which is wildly different from T_A_C @ inv(T_A_B) or T_A_B @ inv(T_A_C) or any other expression you could come up with.

don’t just go by whether the resulting picture looks right or wrong. it might just accidentally look right (but be wrong) because of how the numbers fall (how the dice fall).