How to create a Face Recognition Model using FaceNet Keras?

Image link

Google’s deep convolutional network — FaceNet proposed in 2015 by Google Researchers overcomes the hindrance of implementing face verification and recognition efficiently at scale and achieved the state of the art results. The authors introduced us to the concept of harmonic embeddings, and a harmonic triplet loss, which describes different versions of face embeddings (produced by different networks) that are compatible with each other and allow for direct comparison between each other. FaceNet is the backbone of many opensource systems such as FaceNet using Tensorflow, Keras FaceNet, DeepFace, OpenFace.

Let’s begin talking about what actually face recognition is?

Face recognition is a technique of identification or verification of a person using their faces through an image or a video. It captures, analyzes, and compares patterns based on the person’s facial details. The face detection process is an essential step as it detects and locates human faces in images and videos.

Identification means to compare your identity to all the identities present in the database. Whereas, verification refers to a comparison of your identity to a particular match in the database.

Let’s understand this with an example. Suppose you are entering your office which has a face recognition system. Here, the system will compare your face with all the other faces residing in the company’s database(i.e., One to Many match). This is known as identification. Now, we’ll consider unlocking our devices with face recognition. Here, we are comparing our live face with our stored face in the devices’ database(i.e., One to One match).

The complete Face Recognition system can be divided into three categories :

1. Face Detection

2. Feature extraction

3. Feature Matching

Basically, the output of one method is taken as an input for the next method.

Face Detection

The Face detection method is used to find the faces present in the image, extract the faces, and display it (or create a compressed file to use it further for feature extraction).

Methods used in Face Detection:

1. Haar cascade Face Detection

2. Dlib (HOG) Face Detection

3. Dlib (CNN) Face Detection

4. MTCNN Face Detection

Haar cascade Face Detection: This method has a simple architecture that works nearly real-time on CPU. Also, it can detect images at different scales. But the major drawback is that it gives false results as well as it doesn’t work on non-frontal images.

Dlib (HOG) Face Detection: It is the fastest method on CPU which can work on frontal and slightly no-frontal images. But it is incapable of detecting small images and handling occlusions. Also, it often excludes some parts of the chin and forehead while detection.

Dlib (CNN) Face Detection: It works very fast on GPU and is capable to work for various face orientations in images. It can also handle occlusions. But the major disadvantage is that it is trained on a minimum face size of 80*80 so it can’t detect small faces in images. It is also very slow on the CPU.

MTCNN: This method gives the most accurate results out of all the four methods. It works for faces having various orientations in images and can detect faces across various scales. It can even handle occlusions. It doesn’t hold any major drawback as such but is comparatively slower than HOG and Haar cascade method.

You can refer this site to get the detailed understanding of the above differences.

COMPARISON ( Watch The Difference ):

Haar Cascade

MTCNN

Dlib (HOG)

Dlib (CNN)

Hence, we will be using the MTCNN Face Detection method.

Libraries to install:

1. NumPy

2. Matplotlib

3. PIL

4. MTCNN

5. Sklearn

6. Keras

Link to dataset, compressed files and pretrained Model file.

CODE:

FEATURE EXTRACTION

Feature extraction is the basic and most important initializing step for face recognition. It extracts the biological components of your face. These biological components are the features of your face which differ from person to person. There are various methods which extract various combination of features, commonly known as nodal points. No two people can have all the nodal points similar to each other except for identical twins.

80 Nodal Points(Landmarks)

128 Face Embeddings

Facial Feature extraction has two approaches:

1. Shallow Approach

1. PCA (Dimensionality Reduction Method)

2. LDA (Dimensionality Reduction Method)

3. Cosine Similarity (Feature Matching)

4. HOG

5. SIFT

2. Deep Approach (Industrial Giant Nets)

1. VGG

2. Face Recognition API

3. FaceNet Keras

PCA: PCA is used to reduce the dimensionality of the data. In PCA, the original features of your dataset will be converted into a linear combination of uncorrelated variables(features). These combinations are known as Principal Components. PCA increases algorithm performance and improves visualization. But for reading the PCA components, you need to standardize the data otherwise it won’t be possible to find the optimal Principal Components. Also, PCA results in loss of information if the number of Principal Components is not selected wisely. Learn more

LDA: LDA is a dimensionality reduction technique used to classify different classes based on the features of the supervised data. The major drawback of LDA is the so-called Small Sample Size (SSS) Problem and non-Linearity problem. SSS Problem occurs when the sample (training dataset) is quite small as compared to the dimension of the data.

Cosine Similarity: The measure of cosine angle between two vectors is known as the cosine similarity between two vectors, the closer the cosine value to 1 and greater will be the possibility of a match. One vector among the two vectors is the test data (detected face) and the other is the vector of the training dataset (one of the detected faces of train dataset). But it gives false results for sparse numeric data.

HOG: HOG only uses magnitude values of pixels without including the neighboring values which lead to the extraction of improper features during image rotation.

SIFT: SIFT is relatively alike to a sparse descriptor. Sparse Descriptor is a technique that initially detects the key points in the image and then generates descriptors at these points. SIFT consists of scale rotation and affine transformation properties as well. But it requires a long-running time as compared to other systems.

VGG: VGG uses various architectures such as VGGFace1, VGGFace2 by Keras. The basic difference among these models is the number of layers included in its architecture that varies from model to model. These models have quite a good accuracy. Click here to learn about VGG16 and VGG19.

FaceRecognition API: FaceRecognition API is easier to use. It has a much easier architecture to implement with some inbuilt libraries required for feature recognition. You need to upload a picture and call the FaceRecognition API. The API then simulates the browser using the user’s information to call the recognition points. It works well in real-time and holds good accuracy.

FaceNet Keras: FaceNet Keras is a one-shot learning model. It fetches 128 vector embeddings as a feature extractor. It is even preferable in cases where we have a scarcity of datasets. It consists of good accuracy even for such situations.

We will be using the FaceNet Keras Model.

CODE:

FEATURE CLASSIFICATION

Feature classification is a geometry-based or template-based algorithm used to classify the features of the test data among different classes of facial features in the training data. These template-based classifications are possible using various statistical approaches.

The well-known methods used in feature classification can be given as:

1. Euclidean Distance

2. Cosine Similarity

3. SVM

4. KNN

5. ANN

Euclidean Distance: It is a distance-based feature classification method that calculates the distance between the facial nodes and the face which has the minimum difference between these distance values is considered to be the match. But it is suitable for the datasets having a smaller number of classes and lower dimensionality features.

Cosine Similarity: In cosine similarity, the solution that we obtain after calculating the cosine of an angle is brought into concern. Here, we would compare the differences between these results. The more the value is closer to 1, the greater is the probability of the match. But it may give a false result if the test data features are incomplete (i.e. if the resultant value is 0 then the features don’t match, and if nearly all the features match then the value is 1).

SVM: SVM (Support vector machine) creates an optimal hyperplane to classify the classes of training dataset based on the different features of the face. The dimensionality of the hyperplane is one less than the number of features. Different kernels can be applied to see what features are used by the classifier to remove the features if required. This can help to improve speed. Click here for detailed understanding.

KNN: KNN (K-Nearest Neighbor) is all about the number of neighbors i.e. the k value. In KNN, if k=3 then we check that the data is close to which 3 data points. Thereafter, it is decided that the majority of closest data points belong to which class. Now, the test data is predicted to be in this class KNN has curse of dimensionality problem which can be solved by applying PCA before using KNN classifier. You can get a better understanding of KNN here.

ANN: ANN (Artificial Neural Network) uses a very detailed algorithm for face recognition. It classifies the local texture using multi-layer perceptron for face alignment. It uses geometric feature based and independent component analysis for feature extraction and multi artificial neural network for feature matching. This is the best method to use. You can get a detailed understanding of ANN for Face Recognition in this paper.

Here, we will be practicing the SVM method.

CODE:

from matplotlib import pyplot as plt

from matplotlib.patches import Rectangle

from matplotlib.patches import Circle

from PIL import Image

from numpy import savez_compressed

from numpy import asarray

from os import listdir

from mtcnn.mtcnn import MTCNN

#Method to extract Face

def extract_image(image):

img1 = Image.open(image)            #open the image

img1 = img1.convert('RGB')          #convert the image to RGB format

pixels = asarray(img1)              #convert the image to numpy array

detector = MTCNN()                  #assign the MTCNN detector

f = detector.detect_faces(pixels)

#fetching the (x,y)co-ordinate and (width-->w, height-->h) of the image

x1,y1,w,h = f[0]['box']

x1, y1 = abs(x1), abs(y1)

x2 = abs(x1+w)

y2 = abs(y1+h)

#locate the co-ordinates of face in the image

store_face = pixels[y1:y2,x1:x2]

plt.imshow(store_face)

image1 = Image.fromarray(store_face,'RGB')    #convert the numpy array to object

image1 = image1.resize((160,160))             #resize the image

face_array = asarray(image1)                  #image to array

return face_array

#Method to fetch the face

def load_faces(directory):

face = []

i=1

for filename in listdir(directory):

path = directory + filename

faces = extract_image(path)

face.append(faces)

return face

#Method to get the array of face data(trainX) and it's labels(trainY)

def load_dataset(directory):

x, y = [],[]

i=1

for subdir in listdir(directory):

path = directory + subdir + '/'

#load all faces in subdirectory

faces = load_faces(path)

#create labels

labels = [subdir for _ in range(len(faces))]

#summarize

print("%d There are %d images in the class %s:"%(i,len(faces),subdir))

x.extend(faces)

y.extend(labels)

i=i+1

return asarray(x),asarray(y)

#load the datasets

trainX,trainY = load_dataset('Indian-celebrities/')

print(trainX.shape,trainY.shape)

#compress the data

savez_compressed('Indian-celeb-dataset.npz',trainX,trainY)

Face recognition has many usages such as access control, shopping online, finding lost people, tracking down criminals, etc. But with this comes the requirement of accuracy, surveillance angles which can be effectively fulfilled by Keras FaceNet.

REFERENCES:

[1] https://www.bayometric.com/identification-verification-segmented-identification/

[2] http://blog.altabel.com/2018/08/14/facial-recognition-technology-where-is-it-being-used-and-how-does-the-technology-work/

2

More from Clique Community