https://docs.opencv.org/3.4/de/d25/imgproc_color_conversions.html
Here is that documentation. The subtracting 128 helps obviously (I figured that much out). I will investigate this exact version of XYZ and see if that fits with expected values. They still seemed very high but I will double check.