Face Recognition Methods: Complete Overview

From Antispoofing Wiki

Face recognition is a method of detecting or verifying the identity of an individual using their face. Face recognition is an important part of Antispoofing. This article provides a review of the most effective methods and techniques of face recognition, as well as its primary issues and challenges.

Face Recognition: Effective Methods & Techniques

Face recognition has wide ranging applications: from shopping and entertainment to law enforcement, banking, and surveillance. When used as a tool for biometrical recognition, it should retain a high level of accuracy. This in turn requires integration of a number of highly reliable Face Recognition Methods (FRMs).



The reliability and accuracy of FRMs is especially vital as facial recognition has become an extremely popular feature of mobile gadgets today. According to a study, the face unlock feature will be used by up to 800 million mobile gadgets by 2024. Apart from this, rapidly advancing technology has led to increased use of face recognition in virtual reality, surveillance, security, and other such systems.


Definition

Face recognition methods are a group of techniques that allow detecting and extracting facial features for further analysis.

In essence, face recognition is focused on four primary tasks:

  1. Detection. A target’s face is detected.
  2. Analysis. The system captures the image of the face for further analysis.
  3. Image conversion. Facial features are turned into a faceprint — a set of digital data that "describes" the facial characteristics in the form of digital code, vector data, etc.
  4. Identification. The system matches the captured face with the faces stored in its database.

This detection algorithm is further strengthened using a number of tools including global and local classifiers, principal component analysis, discrete cosine transform, locality preserving projections, and neural networks etc. Methods of facial recognition are based on cross correlation coefficient, subspace, and face geometry.

Methods Concerning Local and Global Features

Local and Global features form a tandem which allows quick and accurate face recognition. It can be used even if an individual’s face is "distorted" by lighting, aging, viewing angle, and obtruding objects like glasses etc.

Below is a more detailed overview of both methods:


Local Features

Local feature analysis employs the Gabor wavelets. It focuses on local information in the high frequency range. The face characteristics analyzed by this method are grouped inside a specific face area: mouth, eyes, cheeks, nose, and so forth. This method involves grouping a number of feature vectors spatially. These groups in turn correspond to facial local patches. A group like this is dubbed Local Gabor Feature Vector (LGFV). LGFV serves as a supplement to the Global feature analysis. It conveys more detailed information about a person’s face, allowing the system to analyze the facial regions.


Global Features

Global feature analysis is focused on the human face as a whole. It takes into consideration general data such as the head contour, facial shape, and so on. It is focused on the low frequency range. Components found in this range are pieced together to form a single feature vector dubbed Global Fourier Feature Vector (GFFV).

Linear discriminant

When the system extracts both global and local features, the Fisher’s linear discriminant is applied to: a) Global Fourier features b) Local Gabor features. The vector data is then fused using a region-based merging algorithm leading to the verification phase which finalizes the recognition process.

Primary Face Recognition Methods

Researchers underline the following face recognition methods:

Appearance-based Method

Appearance based techniques are the oldest used facial recognition method: its roots can be traced back to 1970 when a system designed by a Japanese researcher Takeo Kanade was able to detect facial features and estimate the distance ratio between them.

Modern appearance-based methods do the same, with some additional steps:

  • The feature vectors of the test and database images are obtained.
  • The system tries to detect similarities between the vector images — the minimum distance criterion is one of those.

The appearance-based method has evolved over time and now employs Linear Discriminant Analysis, Principal Component Analysis, Support Vector Machines, Kernel Methods, and so on. Currently, this technique is template-based, meaning that the feature vector represents the entire face template and not just separate facial feature regions.

Subspace-based Method

Subspace-based techniques involves principles such as dimensionality reduction, linear discriminant analysis, and independent component analysis. This method computes a coordinate space, thereby reducing the number of dimensions of a face image. At the same time, it delivers a maximum variance across each orthogonal subspace direction. Subspace-based method helps to detect the vectors that best account for the face image dissemination within the entire image space. It creates a face space — a special group of vectors of reduced dimension. After that linearization, mean vector calculation, covariance matrix and eigenvectors computing, as well as other manipulations are applied to get the end result.

Neural network method

The neural network method employs two networks. The first network functions in the "auto-association" mode. It is used for approximating the face image, which requires a vector with smaller dimensions. The "companion" network plays the role of a classification net — it receives the vector from the auto-association network for analyzing it. Neural networks should be selected individually for each specific situation. A major challenge of this method is that a network requires a large number of model images to be trained with.


Model-based Method

The model-based method is focused on extracting the feature vector. This vector comprises of the face shape, texture, temporal variations, and so on. Moreover, it employs the point distribution model (PDM). It absorbs 134 key points that indicate location of two-dimensional facial features such as mouth, eyes, etc. This method also relies on face localization, texture extraction with the planar subdivisions, texture warping, and image normalization to mitigate the global illumination effects.

Other methods

While the above-mentioned methods can be described either as holistic or feature-based, there’s also a hybrid method in face recognition. The hybrid method commonly relies on 3D images and includes the following stages: Detection, Position, Measurement, Representation and Matching. Since it uses facial depth, the hybrid method can capture and analyze the shapes of nose, forehead, and other features. Interestingly, due to its 3D approach, the hybrid method can even recognize a face in a profile.

Artificial Neural Networks Approaches for Face Detection

Artificial Neural Networks (ANNs) play a significant role in facial recognition. Various methods have been proposed involving multiple neural network types: Deep Convolution, Radial Basis, Bilinear Convolution, and Back Propagation etc.

Two ANN methods deserve a more detailed overview:

MTCNN


Multitask Cascaded Convolutional Network (MTCNN) is a GitHub library powered by the Python language. Inspired by a study on face detection, this network is capable of locating and returning face coordinates or the pixel values of a detected face. Moreover, MTCNN can locate facial "keypoints" such as eyes, nose, chin, and mouth etc. MTCNN represents a detected face in a rectangular box. MTCNN proves to be faster when powered by a GPU and the likes of CUDA: it can process up to 100 images per second.

FaceNet


FaceNet is a deep neural network. Its key advantage is that it can learn mapping directly from face images. It then breaks down an individual’s face image into a vector made of 128 numbers. Such a low-dimensional vector is dubbed embedding: the image data is literally embedded into the vector retrieved by the network. Then, these vectors are interpreted through the Cartesian coordinate system. This allows the mapping of an image in question in the coordinate system by using the embeddings. As a result, the system can compare the embeddings of a person with the embeddings of other people stored in a database. FaceNet requires specific training with the help of anchor images and positive/negative examples. (This method is known as triplet loss).

References

  1. An early example of a face recognition algorithm
  2. Biometric facial recognition hardware present in 90% of smartphones by 2024
  3. We fooled the OnePlus 6's face unlock with a printed photo
  4. Scheme of the Global and Local feature analysis
  5. Gabor wavelets
  6. Local Gabor Feature Vector (LGFV)
  7. Gabor wavelet used in a facial recognition algorithm
  8. Fisher’s linear
  9. A system designed by a Japanese researcher Takeo Kanade
  10. Face Recognition for Beginners
  11. Face Detection and Recognition Theory and Practice eBookslib
  12. A neural network used in face recognition and antispoofing
  13. The model-based method
  14. Face Recognition Methods & Applications
  15. An example of MTCNN performance
  16. GitHub library
  17. A study on face detection
  18. Face Detection using MTCNN — a guide for face extraction with a focus on speed
  19. Face image compression by FaceNet
  20. How to Develop a Face Recognition System Using FaceNet in Keras
  21. FaceNet: A Unified Embedding for Face Recognition and Clustering