From rectified camera coordinate system to unrectified

After stereo calibration I get the rectified parameters using stereoRectify()
Using Q and the disparity map I can get 3D points.
How to convert the 3D points to the original camera CS? I need a R and T from rectified to normal. I think R = inv(R1) from stereoRectify().