DNN based Face Recognition isn't working - Euklidian distance is always 0 [Solved]

C++
,
1

I want to verify if one person is similar to another person. Therefore I want to get the similarity between two faces.

These are the input faces

Mindy Face

Madonna Face

Now I want to push them through the DNN and after that I want to get the Euklidian distance between the two resulting matrices.

I’ve used the following OpenFace model: openface-models/nn4.small2.v1.t7

This is my code for calculating the distances:

    cv::Mat madonna = cv::imread("/home/der/Madonna_Face.jpg");
    cv::Mat mindy = cv::imread("/home/der/Mindy_Face.jpg");

    cv::resize(madonna, madonna, cv::Size(96, 96));
    cv::resize(mindy, mindy, cv::Size(96, 96));

    cv::Mat madonnaBlob = cv::dnn::blobFromImage(madonna, 1.0 / 255, cv::Size(96, 96), cv::Scalar{0,0,0}, true, false);
    cv::Mat mindyBlob = cv::dnn::blobFromImage(mindy, 1.0 / 255, cv::Size{96, 96}, cv::Scalar{0,0,0}, true, false);

    cv::dnn::Net _net = cv::dnn::readNetFromTorch("/home/der/nn4.small2.v1.t7");

    _net.setInput(madonnaBlob);
    cv::Mat madonnaMat = _net.forward();

    _net.setInput(mindyBlob);
    cv::Mat mindyMat = _net.forward();
    
    std::cout << cv::norm(madonnaMat, mindyMat);

And if I’m doing so the result from cv::norm is 0.
The representations are exactly the same:

    std::vector<double> master = madonnaMat;
    std::vector<double> slave = mindyMat;
    for(int i; i < 128; i++) {
        std::cout << master[i] << " # " << slave[i] << std::endl;
    }


Output:
    > -0.0865457 # -0.0865457
    > 0.133816 # 0.133816
    > -0.105774 # -0.105774
    > 0.05389 # 0.05389
    > -0.00391233 # -0.00391233
    > ...

Results:

Madonna Representation: [-0.060358506, 0.14156586, -0.10181303, 0.060315549, 0.0016125928, 0.066964693, -0.044892643, -0.043857966, 0.088271223, 0.047121659, 0.078663647, 0.025775915, 0.062051967, 0.034234334, -0.049976062, 0.045926169, 0.084343202, 0.046965379, -0.092582494, 0.13601208, -0.003582818, -0.15382886, 0.075037867, 0.19894752, -0.041007876, -0.12050319, -0.056161541, -0.018724455, 0.024790274, 0.0092850979, 0.095108159, 0.067354925, 0.06044127, 0.041365273, -0.12024247, 0.18279234, 0.027767293, 0.09874554, -0.16951905, 0.062370241, -0.014530737, 0.015518869, -0.0056175897, -0.066358574, -0.02390888, -0.07608442, 0.13011196, 0.031423025, -0.010443882, 0.12755248, -0.010195011, 0.0051672528, -0.10453289, -0.013270194, 0.096139617, 0.10375636, -0.047089621, 0.050923191, 0.066422582, -0.046726897, -0.1845296, 0.031028474, 0.086226918, -0.27064508, 0.055891197, -0.0053421594, 0.035870265, -0.026942547, -0.17279817, 0.13772435, 0.0071162563, 0.075375959, -0.046405111, 0.12658595, 0.11093359, 0.0030428318, 0.070016958, 0.1725318, -0.056130294, -0.14420295, -0.12438529, 0.056423288, -0.080888703, -0.052004829, -0.06481386, 0.14163122, -0.059617694, -0.026075639, 0.052098148, -0.0055074869, -0.014869845, -0.11943244, 0.068051606, -0.096071519, 0.19727865, -0.016027609, -0.05776047, 0.069935486, -0.020494614, 0.013407955, -0.06065508, -0.056143567, -0.04608072, 0.072748154, -0.035580911, 0.15261506, -0.074352823, -0.081481896, 0.020475708, -0.021581693, -0.16350025, 0.12794609, 0.082243897, 0.015881324, 0.011330541, -0.026391003, 0.086644463, -0.10490314, 0.088207267, 0.17892174, 0.025871141, 0.012454472, 0.010682535, 0.1253885, -0.12909022, 0.082067415, -0.035789803, 0.032903988]
Madonna Size: 1 x 128
Madonna Dims: 2
Mindy Representation: [-0.082645342, 0.14463238, -0.10716592, 0.065654278, 0.0045089996, 0.064019054, -0.047334831, -0.056190431, 0.099919245, 0.048234992, 0.068906084, 0.028518379, 0.057044145, 0.046223734, -0.056203742, 0.033566523, 0.082230642, 0.055683684, -0.080982864, 0.12431844, -0.00075431512, -0.14511517, 0.045022864, 0.20965824, -0.030178605, -0.11852413, -0.066858761, -0.01461118, 0.032898057, 0.02857255, 0.1088237, 0.07066118, 0.044605579, 0.022743503, -0.10785796, 0.20373915, 0.010565795, 0.063950166, -0.18701579, 0.062780239, -0.0042907735, 0.031276166, -0.006556896, -0.038440779, -0.01419229, -0.072688736, 0.13676986, 0.040385362, 0.010314438, 0.095734902, -0.0080824783, 0.011763249, -0.098884396, -0.040797569, 0.10534941, 0.12088351, -0.07317061, 0.063644305, 0.0830286, -0.050620016, -0.18088549, 0.03330183, 0.090282671, -0.25393733, 0.056058947, -0.020288708, 0.049997903, -0.044997148, -0.15860014, 0.15251927, 0.015151619, 0.088731326, -0.028061632, 0.11127418, 0.090425298, 0.0052096732, 0.053858042, 0.18543676, -0.066999368, -0.15851147, -0.11389373, 0.088093147, -0.08713299, -0.048095752, -0.063261949, 0.12453313, -0.051213119, -0.023759408, 0.048403475, -0.012721839, -0.021282939, -0.098075315, 0.066707589, -0.11601795, 0.20438787, -0.015739718, -0.052848384, 0.057336167, -0.01592578, 0.014057826, -0.058749981, -0.043632519, -0.031006066, 0.046038814, -0.065755703, 0.15442967, -0.082077362, -0.099808514, 0.016168201, 0.0046916353, -0.14556217, 0.11152669, 0.062443323, -0.00032889194, 0.0020548289, -0.026999777, 0.096809812, -0.11947374, 0.085579365, 0.16317753, 0.028130196, 0.014577032, 0.0079531483, 0.11340163, -0.15006165, 0.094127603, -0.0440454, 0.033095147]
Mindy Size: 1 x 128
Mindy Dims: 2

Any ideas what I’m doing wrong? Thanks.

2

you’re actually very close :slight_smile:
net.forward() returns a reference to the output layer,
so unless you clone() that (to get a deep copy),
2 sequential predictions will only return the last result.
solution:

besides that, the dnn works on float32 data, not double, so, fix type here:

    std::vector<double> master = madonnaMat;
2 Likes
3

related: