i guess, your model was trained on rgb images
(and now you feed it a thresholded 1 channel image)
can you show one of your train images ?
(if those are tightly cropped, you probably should crop the (bgr !) frame to the hand rect, and feed that into your nn)