fitEllipse doesn’t find the minimal enclosing ellipse/circle. it will actually try to fit an ellipse through the contour points you give. if your contour is that noisy, it’ll do what you see.
your segmentation needs improvement. I’d say different threshold, vary the morphology operations (no opening, perhaps just a dilation).