Any tips on confidence score for face verification (as opposed to face recognition)?

Question

I'm using eigenfaces (PCA) for face recognition in my code. I used the tutorials in OpenCV's website as a reference. While this works great for recognizing faces (ie it can tell you who is who correctly), the confidence-score based face verification (or imposter detection- verifying whether the face is enrolled in the training set) doesn't work well at all.

I compute a Euclidean distance and use it as a confidence threshold. Are there any other ways I could calculate a confidence threshold? I tried using Mahalanobis distance as mentioned in http://www.cognotics.com/opencv/servo_2007_series/part_5/page_5.html , but it was producing pretty weird values.

PS: Solutions like face.com won't probably work for me because I need to do everything locally.

Answer 1

You can project the new input face onto the Eigenspace using subspaceProject() function and then generate the reconstructed face back from the Eigenspace using subspaceReconstruct() and then compare how similar the input_face and the reconstructed_face is. A known face (face in the training data set) will have the reconstructed image more similar to the input_face than an imposter's face. You can set a similarity threshold for verification. Here's the code:

// Project the input face onto the eigenspace.
Mat projection = subspaceProject(eigenvectors, FaceRow,input_face.reshape(1,1));

//Generate the reconstructed face
Mat reconstructionRow = subspaceReconstruct(eigenvectors,FaceRow, projection);

// Reshape the row mat to an image mat
Mat reconstructionMat = reconstructionRow.reshape(1,faceHeight);

// Convert the floating-point pixels to regular 8-bit uchar.
Mat reconstructed_face = Mat(reconstructionMat.size(), CV_8U);

reconstructionMat.convertTo(reconstructed_face, CV_8U, 1, 0);

You can then compare the input face and the reconstructed face using cv::norm(). For example:

// Calculate the L2 relative error between the 2 images. 
double err = norm(input_face,reconstructed_face, CV_L2);
// Convert to a reasonable scale
double similarity = error / (double)(input_face.rows * input_face.cols);

Answer 2

You can look at feature extraction algorithms other than PCA like LDA (Linear Discriminant Analysis) or Local Binary Patterns (LBP).

LDA models interclass variation and LBP is an illumination invariant descriptor. You have the implementation of both these algorithms in OpenCV. Check the below link.

http://docs.opencv.org/trunk/modules/contrib/doc/facerec/facerec_api.html