Impact of Aging on Face Recognition: Studies, Databases and Recognition Techniques

From Antispoofing Wiki

General Overview

Aging is a natural physiological process that occurs in a human organism during its growth. Face is a region where aging effects are more apparent when compared to rest of the body. Apart from natural growth, the facial structure can also be impacted by age-related weight gain or loss, dietary habits, location and gravity, smoking or drinking alcohol, substance abuse, sunlight exposure, hormonal imbalance, loss of teeth, and other reasons.



This creates an obstacle for facial recognition systems, as the human face has the lowest score in terms of permanence — an essential biometric identifier. At the same time, facial recognition is seen as the most preferable modality because of its contactless, reliable and un-intrusive nature if working in the passive regime.



In light of the challenges posed by aging, a need for robust algorithms, that can identify a person despite visual changes in their appearance, has emerged. A number of approaches have been proposed along with datasets and experiments to address this issue.

Statistics & General Information

An extensive research conducted at the Michigan State University involved a face recognition algorithm and two mugshot databases to identify any relationship between facial aging and recognition accuracy. The results showed that a time interval of 6 years is optimal for a system to identify an individual: the recognition accuracy rate was 99%.



However, the success rate of recognition can vary from person to person, as genetics, lifestyle habits and other factors can stimulate or impede facial aging. Accuracy rates would also diminish if test images offered for recognition were captured over a 6-year interval. Interestingly, criminal mugshot databases are a favorable source of sample data: they have a large selection of photos, subjects and time intervals.

Another research at Purdue University demonstrated that False Rejection Rate (FRR) had a persistent growth if a face recognition algorithm was presented with photos that had a time gap among them.

Age-invariant Face Recognition Methods

Age-invariant face recognition refers to techniques that are capable of maintaining accuracy despite noticeable appearance alterations caused by aging. They are categorized as a) Generative — they simulate face changes caused by aging b) Discriminative — they focus on specific features that stay unchanged over time c) Deep learning-based — they involve neural networks.

Generative Methods

A method proposed comprises generative and discriminative aging models. Discriminative model focuses on notable facial features, which make recognition possible despite time lapses. And the generative model is responsible for learning the parametric aging model in the 3D domain to synthesize images where the potential age gap is being removed. (3D modeling fits for detecting aging patterns as aging is a three-dimensional phenomenon.)



An alternative method suggests recreating 3D faces from 2D photos to obtain embracive craniofacial data. It includes stages such as 2D facial feature point detection with the Active Appearance Model (AAM), 3D model fitting through Blanz and Vetter’s model and 3D facial shape represented through eigenvectors, Tikhonov regularization controlling the Mahalanobis distance, pose correction, shape and aging texture patterns, and finally aging simulation. The solution was tested with three datasets — FG-NET, MORPH and Browns — showing improved performance when recognizing age-affected images.





An alternative method is based on the Temporal Non-Volume Preserving (TNVP) transformation, which is capable of breaking down the entire aging process into temporally shorter phases. It’s capable of registering non-linear age variations and modeling a smooth synthesis in age progression.

Non-Generative Methods

To an extent, facial mark analysis can be attributed to the non-generative approaches. It focuses on the local facial marks that retain unique characteristics and have a certain degree of permanence: freckles, lip creases, birthmarks, scars, and so on. They are frequently used as identifiers in forensics. However, plastic surgery or makeup can undermine recognition.



Another method suggests usage of the Gradient Orientation Pyramid (GOP), which can recreate features that are immune to age-induced alteration. It also employs Support Vector Machines (SVM) used for calculating image difference pairs from the cosine between gradient orientations at all scales.

An alternative approach dubbed Hidden factor analysis implies that the face can be divided into two regions: the first region represents the facial ‘fundament’, which remains unchanged for the most part during a person's life. The second region represents the combined aging effects that occur along with the anatomical growth. Expectation–Maximization procedure is used for estimating the latent factors.

A demography-based study has also been suggested. It assumes that facial aging can be better understood through the prism of demographic estimation of the human face. The study focuses on facial symmetry, which plays a pivotal role in the aging mechanism.

Deep Neural Network Approaches

Convolutional Neural Networks are proposed for the age-invariant face recognition. One of the concepts directly advises training a CNN to detect jointly features, distances and thresholds that occur together with age-related alterations. A different method is a set-based approach, in which photos of a certain subject are put in a set of images. Then, this set is compared to image sets of other individuals, while feature extraction is conducted with a CNN.

Finally, it is recommended to fuse generative and discriminative approaches. That implies that distinctive features of age-invariant and age-sensitive regions should be separated with a CNN. Then, pixel mean vector, local binary patterns, generative model, and bridged denoising auto-encoders are used to process the mentioned features.

Databases

Datasets used for age-invariant face recognition are longitudinal. That means they include an assortment of samples collected over a long time interval from the same group of subjects. Forensic mugshot datasets are often the primary data source in this case.

MORPH

MORPH is, by far, the largest longitudinal dataset with 400,000+ sample images obtained from 70,000 people. It’s periodically renewed and also supported with the metadata for each individual: gender, weight, height, etc. There are two versions: corporate and academic with 50,000+ samples from 13,000+ subjects and with the biggest time interval between photos being 1,681 days (4+ years).


FG-NET

FG-NET offers 1,002 images taken from 82 people. Apart from austere sample variety, it also offers images of average quality at best, which can be a favorable option for training a solution that is fit for ‘uncontrolled environments’. Currently, it may be unavailable from the official distributor. Alternative download links are here and here (courtesy of Yanwei Fu).


PCSO_S and LEO_LS

LEO_LS includes 31,852 images of 5;636 people with the biggest time interval of 8 years. PCSO_S contains 147,784 operational mugshots captured from 18,007 repeat criminal offenders. The photos were shot in the time period from1994 to 2010. Currently, both datasets seem to be unavailable.


NIST Special Database 18

This dataset is not exactly longitudinal. However, it features 3,248 images taken from 1,573 individuals posing for mugshots in front view and profile. Potentially, the dataset can be used for training a generative face recognition method.

Solutions & Experiments

A number of pre-trained CNNs — Alex-Net, GoogleNet, Inception V3, ResNet50 and SqueezeNet — were used for testing an algorithm capable of feature extraction for age-invariant face recognition. As it turned out, a descriptor Fc7 supported with an SVM classifier achieved a 98.21% accuracy rate. AlexNet, also combined with an SVM classifier, showed the highest performance score with a 98.21% accuracy rate.


References

  1. Factors influencing face aging. Literature review
  2. A series of photos demonstrating face aging of a single person
  3. Year in Review: COVID Shakes Up the Top Biometric Modality Rankings
  4. Aging faces could increase security risks
  5. Sample from the NIST Special Database 18
  6. Aging effects in automated face recognition
  7. Face recognition: Some challenges in forensics
  8. Age-Invariant Face Recognition
  9. Active Appearance Model by Wikipedia
  10. A Morphable Model for the Synthesis of 3D Faces
  11. Tikhonov Regularization and ERM
  12. Mahalanobis Distance – Understanding the math with examples (python)
  13. FG-NET
  14. MORPH
  15. Active Appearance Models in C++ (Paamela)
  16. Deep-learning based descriptors in application to aging problem in face recognition
  17. Temporal Non-Volume Preserving Approach to Facial Age-Progression and Age-Invariant Face Recognition
  18. Detection and matching of facial marks in face images
  19. Examples of facial marks Scar, mole, and freckles etc.
  20. Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments
  21. A Gentle Introduction to Expectation-Maximization (EM Algorithm)
  22. MORPH Facial Recognition Database
  23. academic
  24. MORPH dataset sample
  25. FG-NET dataset by Yanwei Fu
  26. PCSO_S database sample
  27. NIST Special Database 18
  28. AlexNet: The Architecture that Challenged CNNs