that’s how optical flow works. there is no deeper state or history or “model” of these points. it’s just points on one frame, the local neighborhood for each point in that one frame, and their best matches on the next frame.
you shouldn’t ask about implementation first. you should just assume that any implementation sticks to the original paper, and any deviations would be documented.