Index of corners in contour change when shape is rotated

Hello,
As you can see in the following images:

1.PNG (gyazo.com)
2.PNG (gyazo.com)

the corners of the triangle flip when the shape is rotated. This is explained with this:
Y4dPZ

Basically, the index of the corner switches depending on which point has the highest value in y-axis.

How can I work around this? I need to fix the points so that they’re unchanged even when rotated.

The code I used to extract those coordinates is:

                    p2 = cv2.arcLength(cnt, True)
                    # appr contains the 4 points
                    appr = cv2.approxPolyDP(cnt, 0.02 * p2, True)
                    appr = sorted(appr, key=lambda c: c[0][0])
                    # pa = top left point, pb = bottom left point
                    pa, pb = sorted(appr[:2], key=lambda c: c[0][1])

                    # pc = top right point, pd = bottom right point
                    pc = sorted(appr[2:], key=lambda c: c[0][1])

                    # The points are x, y in list of list [[x, y]]
                    xa = pa[0][0]
                    ya = pa[0][1]
                    xb = pb[0][0]
                    yb = pb[0][1]
                    xc = pc[0][0][0]
                    yc = pc[0][0][1]

related:

you can’t distinguish this…

image

from this, which is rotated another half turn:

image

so there’s ambiguity already.

if you want to use the fact that it’s a rectangle, one side is longer than the other, you can… just reorder the points.