Converting UMat to Mat data type takes too long. How to solve this problem

I find that when using the filter2D function, the larger the convolution kernel, the longer it takes to convert from UMat to Mat after filter2D convolution. Why? These are the results of my experiments
After 13*13 convolution, it takes about 58ms for UMat to Mat
image
After 11*11 convolution, it takes about 37ms for UMat to Mat
image
After 9*9 convolution, it takes about 19ms for UMat to Mat
image
After 7*7 convolution, it takes about 9ms for UMat to Mat
image
After 5*5 convolution, it takes about 7ms for UMat to Mat
image