I wrote a python script using opencv to compare 7k pictures amongst themselves and flag duplicates, but even on my PC:
6 core AMD Ryzen 5 3600X (12 logical ones),
Win10,
16GB RAM,
1TB SSD,
it takes too long. I added threading to make it go faster but even so it took like 10 hours to check 49 pictures against the 7k images. So now I am onto the next idea to speed it up by recompiling opencv to use my nvidia GTX 1660 Ti cause it has more cores. I am using this guide here but with more recent versions: CUDA Toolkit 11.8 and OpenCV 4.7.
I got to step 12.6, after using cmake to do the config and building the ALL_BUILD:
12.6. Right-click “ALL_BUILD ” and click build .
This step fails with the error DIFFERENT_SIZES_EXTRA: undeclared identifier.
INSTANTIATE_TEST_CASE_P(CUDA_Warping, Remap, testing::Combine(
ALL_DEVICES,
DIFFERENT_SIZES_EXTRA, // this line here!!!
testing::Values(MatType(CV_8UC1), MatType(CV_8UC3), MatType(CV_8UC4), MatType(CV_32FC1), MatType(CV_32FC3), MatType(CV_32FC4)),
testing::Values(Interpolation(cv::INTER_NEAREST), Interpolation(cv::INTER_LINEAR), Interpolation(cv::INTER_CUBIC)),
testing::Values(BorderType(cv::BORDER_REFLECT101), BorderType(cv::BORDER_REPLICATE), BorderType(cv::BORDER_CONSTANT), BorderType(cv::BORDER_REFLECT), BorderType(cv::BORDER_WRAP)),
WHOLE_SUBMAT));
I am wondering if I can just comment out these two test cases?
I wouldn’t as it looks like you are mixing up versions of the OpenCV main and contrib repositoriess.
DIFFERENT_SIZES_EXTRA
was added to the main repository in
opencv:4.x
← cudawarped:fix_issue_3412
opened 04:28PM - 10 Jan 23 UTC
Add single dimension `GpuMat` testing define because single dimension `GpuMat`'s… memory is allocated differently.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
so if you have the test cases from
opencv:4.x
← cudawarped:fix_issue_3412
opened 04:18PM - 10 Jan 23 UTC
This fixes issue https://github.com/opencv/opencv_contrib/issues/3412 returning … incorrect results for single `GpuMat`'s with a single row, which was a result of an error in https://github.com/opencv/opencv_contrib/pull/3378.
Unfortunatley because `GpuMat`'s with a single row or column are allocated with `cudaMalloc` and not `cudaMallocPitch` and 2D texture objects require pitched memory this PR introduces extra staging memory, when the row/col dimension is 1. This is a quick fix for the issue which only introduces extra overhead for single dimension `GpuMat`'s who's use should not be that common.
A "better" fix would be to rework the `TexturePtr` class so that
```
__device__ __forceinline__ R operator ()(index_type y, index_type x) const;
```
staticly dispatches a call to `tex1Dfetch` when either x or y is 1, but this will be a more significant change.
Dependant on https://github.com/opencv/opencv/pull/23126
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
```
force_builders=Custom
buildworker:Custom=linux-1
docker_image:Custom=ubuntu-cuda:16.04
```
but not the defines you are using a more recent commit from the contrib than the main repo.
I would rebuild from the tip of the master of both the main and contrib repositories.
berak
February 22, 2023, 7:44am
3
maybe you dont even need gpu support for your task, but a less naive algorithm !
say, you have 40 new images, and want to find out, if those are already in your 7k database:
use phash, to generate 64bit (8 byte) signatures from your database images. (this will take a while, but you only need to do that once ) put those into a dict with sig as key, filepath as value & serialize to disk.
for new images, get a phash signature again, compare that to those in the dict (a matter of microseconds), if it’s already there, discard, else save image to database folder & append sig/filepath to dict (and serialize dict again)
see, i’m regularily checking my webbrowser cache for images (fodder for machine learning), extracting like 1k images from there, and check those against a ~200k (and ever growing !) image database, works like a charm !
https://docs.opencv.org/4.x/d4/d93/group__img__hash.html
note, that phash will also work with cropped / resized / gray / flipped / rotated images, unlike your naive subtraction !
oh nice thanks, yeah the 7k is just the sample, i got a much bigger set to compare with later
Same problem
INSTANTIATE_TEST_CASE_P(CUDA_Warping, Remap, testing::Combine(
ALL_DEVICES,
DIFFERENT_SIZES_EXTRA, // this line here!!!
testing::Values(MatType(CV_8UC1), MatType(CV_8UC3), MatType(CV_8UC4), MatType(CV_32FC1), MatType(CV_32FC3), MatType(CV_32FC4)),
testing::Values(Interpolation(cv::INTER_NEAREST), Interpolation(cv::INTER_LINEAR), Interpolation(cv::INTER_CUBIC)),
testing::Values(BorderType(cv::BORDER_REFLECT101), BorderType(cv::BORDER_REPLICATE), BorderType(cv::BORDER_CONSTANT), BorderType(cv::BORDER_REFLECT), BorderType(cv::BORDER_WRAP)),
WHOLE_SUBMAT));
Then the solution is probably
I wouldn’t as it looks like you are mixing up versions of the OpenCV main and contrib repositoriess.
DIFFERENT_SIZES_EXTRA was added to the main repository in
so if you have the test cases from
but not the defines you are using a more recent commit from the contrib than the main repo.
I would rebuild from the tip of the master of both the main and contrib repositories.