Resize image/video to 2 different resolutions and reduce from RGB24 to the SAME 64 colors for both resolutions

At the moment, I use cv::resize(...,cv::INTER_LANCZOS4) then kmeans() to display RGB24 image or videos on my 128x32 dot matrix display for pinball machines with 64 colors. The result is amazing, thanks!
Now, I would need to resize to 2 low resolutions (128x32 and 256x64) but with the EXACT SAME 64-color palette.
How would you do that keeping the amazing good quality of what I get for one resolution, please?
Thank you