Face Morphing Detection — Necessity, Techniques and Potential Challenges
Face morphing is a spoofing technique, which blends two or more faces into one, allowing fraudsters to deceive a face recognition system.
Definition & Overview
Face morphing is a technique that blends or morphs multiple faces into a single image. This face manipulation technique is widely used by malicious actors to bypass security systems for accessing restricted areas, protected data, illegally crossing state borders, and so on. Previously, face morphing had been used in art and entertainment, as well as in forensics. In 2001 it was proposed as a method to re-create a more accurate portrait of a suspect’s appearance, based on the verbal description of several witnesses.
It is unclear when face morphing became a security threat. One of the early reports was made at the University of Bologna in 2014, mentioning a "magic passport" concept. According to this idea, a "wanted criminal" can avoid challenges at border crossings or airports by digitally merging their face with the face of an accomplice, printing the photo and placing it on a fake ID. According to the authors, this has become possible especially after the introduction of Automatic Border Control systems (ABCs).
Another concern surrounding face morphs states that a perpetrator can use a morphed photo to get identified and authorized as their victim, leading to a number of criminal schemes. Due to its rising criminal threat, face morphing has been receiving attention from researchers and authorities. In 2018, a German activist managed to obtain a government-issued passport, which contained his photo morphed with a face of an Italian politician. Two years later, Germany officially banned "photo morphing in passports", forcing all applicants to either take photos at the passport office or use a secured submission network.
Another concern is the easiness of producing a morphed image today. Currently, there are plenty of publicly available tools, which allow nearly spotless face blending: Photoshop, Face Swap Online, FaceMorpher, Magic Morph, and others. Facial morphing detection, has therefore emerged as a countermeasure to this threat, and allows detecting blended or morphed face images.
Morphed Face Image Manual Detection
Experts note that spotting a face morph with a naked eye is challenging, if not impossible at all. After blending is complete, images regularly undergo post-processing, which is meant to erase all visible artifacts left by the morphing procedure: skin color inconsistencies, layering clues, warping, distortion, unnatural face toponymy, and so on.
The challenge of detecting morphed faces is much harder when it comes to identification documents. As noted in a study, face morphs for ID documents are mostly printed on small-sized pieces of photographic paper — the typical ID photo size is 3.5 x 4.5 cm or 1.3 x 1.7 inches. So, even if there are visible artifacts, a human observer will most likely fail at Morphed Attack Detection (MAD).
Nonetheless, experts have proposed a few techniques that may allow manual Morphed Attack Detection:
Blind/Referenceless Image Spatial Quality Evaluator or BRISQUE proposes a default model for automatic evaluation of image quality. As a study reports, the model shows a high value for bona fide images, meanwhile morphs, even if uncompressed and of irreproachable quality, show a lower evaluation.
Artifacts usually appear when a morphed image lacks precision in placement of landmarks, such as ears, head contour, iris, etc. As a result, a morphed image may include artifacts such as the so-called "ghost artifact". Shadow anomalies, blur, graininess, overly hard edges, unnatural color gradients, and other artifacts also indicate a morph.
When images of people from different ethnicity, age, or gender groups are morphed, the resulting image can have a certain "unnaturalness" and low plausibility. An example is a morphed image containing people who have a considerable age gap.
A proposed method for better manual detection is that personnel should receive instruction and training regarding the MAD. A study proves that without prior knowledge of the morph scenarios, a human operator would not recognize 68% of morphs.
Databases on Morphing Attack Detection
A number of morphed image datasets exist, with some public and most in the private domain.
- The first known dataset was proposed along with the "magic passport" concept. It contains 14 morphs based on 8 bona fide images and created with the GIMP GAP method. This database is exclusively digital and is not available to a broad audience. The same team later introduced two more datasets: one made with Sqirlz and another with triangulation using the dlib landmark method.
- A bigger dataset was developed by Indian researchers. It was created with the GIMP GAP, adding the GNU image manipulation. It has a big sample collection of 450 images, featuring ethnic diversity. Later the same researchers proposed a dataset made with the OpenCV tool.
- A dataset developed by Makrushin et al., was made with complete and splicing morph methods. The former includes facial geometry of the source images, offering 1,326 samples. The latter clips out the "face pixels" out of the input image, offering 2,614 samples.
- Scherhag et al., developed the first dataset to feature physical (print/scan) morphs. Produced with GIMP GAP, it has 231 morph samples.
- The first public dataset was developed by Biometrix. It is based on the NIST FERET database and offers 1,082 samples.
- FRLL-Morphs by IDAP is another publicly available morph dataset. It was created with WebMorpher, StyleGAN2, OpenCV and FaceMoprher, thus offering 4 types of sample material.
The number of facial morphing databases is still low compared to other technologies. As a result, the lack of training material greatly hinders the development of morph detection tools.
Face Morphing Attack Detection
Morphed Attack Detection (MAD) includes four stages:
In this stage, the visual data — like a grayscale Trusted Live Capture (TLC) image — is normalized in terms of size, subject’s pose, face cropping, and other variables. This is necessary to ensure effective analysis. The dlib algorithm is often used for locating facial landmarks.
Next, a feature vector is calculated. This is necessary for extracting spatial data and other vital parameters. For example, an image can be divided into cells for retrieving and connecting histograms. Besides, three types of descriptors can be used at this step:
- Learned by a deep neural network
Texture descriptors can reveal artifacts left by morphing: landmark overlapping, iris artifacts, etc. Gradient-based descriptors help extract the Histograms of Oriented Gradients. Deep Convolutional Neural Networks allow extracting features that are vital for statistical analysis.
The retrieved feature vectors need to be prepared to allow classifier training. Classifiers may require normalized data with multidimensional characteristics, and so on, depending on their type. Therefore, feature preparation is an essential step.
Finally, classifiers are trained based on the previously prepared data.
Face demorphing is an alternative method to MAD, and consists of two steps:
- A person’s live image is compared to the Electronic Machine Readable Travel Document (eMRTD).
- The suspected morph image featured in the ID is reverted to detect the owner’s real identity.
As a result, the system can automatically raise an alarm if attacked. The tests were orchestrated with the help of four SDKs successfully examined during the Face Recognition Vendor Test (FRVT).
Experiments & Evaluation of Methods
Various tests were held to attest MAD solutions. For example, one of the tests showed that error rate becomes much higher if low-quality, grayscale images are used for evaluation. (EER 17% if compared to EER 2.5% from the high-quality image test.)
Cross-dataset examination also shows poorer results, which indicates that MAD datasets contain highly specific parameters. At the same time, mixed-dataset testing shows better results with the error rate of just 35%.
It is suggested that ISO/IEC 30107-3 standard with its APCER/BPCER rates can be used for evaluating MAD solutions. As a result, Face Recognition Vendor Test by NIST offers its own accuracy rating for morph detection based on these metrics.
- Face morphing could catch criminals
- The magic passport
- Face morphing threat to biometric identity credentials’ trustworthiness a growing problem
- Germany bans digital doppelganger passport photos
- Face morphing tutorial in Adobe Photoshop
- Morph Thing allows anyone to easily morph two faces into one
- Morph Creation and Vulnerability of Face Recognition Systems to Morphing
- Face Recognition Systems Under Morphing Attacks: A Survey
- Fraudulent ID using face morphs: Experiments on human and automatic recognition
- Face Morphing Attack Generation & Detection: A Comprehensive Survey
- Facial landmarks with dlib, OpenCV, and Python
- The first public dataset
- NIST FERET database
- FRLL-Morphs is a dataset of morphed faces based on images selected from the publicly available Face Research London Lab dataset.
- Sample from Biometrix’s dataset
- Face Morphing Attack Detection Methods
- Face demorphing in the presence of facial appearance variations
- Face Recognition Vendor Test
- Practical Evaluation of Face Morphing Attack Detection Methods
- FRVT MORPH