How to find the external points of a square/rectangle contour?

So you are telling me that looping through the contours to find the max/min points are the correct method of finding the coordinate points making up of the contours?