I’d suggest debugging the yield of feature descriptors (how many do you get) and the matching step (how many matches).
also PLEASE use a decent descriptor such as AKAZE or SIFT.
make sure you use Lowe’s ratio test to filter the matches. you’ll find that in one of the tutorials in the docs: OpenCV: Feature Matching