Same code running in Linux much slower than in Windows

I have a dual boot PC with Windows 10 and ubuntu 20.04 on same SSD. I build OpenCV 4.5.5 with contrib model in both platforms using same configuration. My aim is to compare the performance of c++ program using OpenCV ximgproc functions in both platforms. To my surprise, the code is running much faster in Windows 10 than in Linux (almost twice as fast). Do you know anything I can tune the Linux build so I can get similar performance?

On Windows 10, I use Visual Studio 2019, with latest updates. On ubuntu, I use the default g++ 9.4 from repos. The test code that I use is as follows (remove all error checking for simplicity):

try {
	cv::Mat h_src = cv::imread(image_path, cv:::IMREAD_GRAYSCALE);
	cv::Mat img_thres_cmb;
	int blockSize = 95, k = 1.5;
	int type = cv::THRESH_BINARY_INV;
	int method = cv::ximgproc::BINARIZATION_WOLF; 

	auto t3 = cv::getTickCount();
	cv::ximgproc::niBlackThreshold(h_src, img_thres_cmb, 255, type, blockSize, k, method);
	auto t4 = cv::getTickCount();
	auto cpu_sgmnt = (t4 - t3) / cv::getTickFrequency();
	std::cout << "[cpu_sgmnt]: [ " << cpu_sgmnt << " ] secs." << std::endl;
catch (const cv::Exception& ex) {
	std::cout << "Error: " << ex.what() << std::endl;

The c++ code is very simple, it basically calls the niBlackThreshold function which is compiled by OpenCV’s build system. I get the exact same result from both platforms, the only issue I have now is that the Linux version is running too slow.

You input and help is greatly appreciated.

Thanks and with Best Regards