Definition & Overview
Facial morphing is a technique that blends or morphs multiple faces into a single image. This face manipulation technique is widely used by malicious actors to bypass security systems for accessing restricted areas, protected data, illegally crossing state borders, and so on. Previously, facial morphing had been used in art and entertainment, as well as in forensics. In 2001, it was proposed as a method to re-create a more accurate portrait of a suspect’s appearance, based on the verbal description of several witnesses.
It is unclear when facial morphing became a security threat. One of the early reports was made at the University of Bologna in 2014, mentioning a "magic passport" concept. According to this idea, a "wanted criminal" can avoid challenges at border crossings or airports by digitally merging their face with the face of an accomplice, printing the photo and placing it on a fake ID. According to the authors, this has become possible especially after the introduction of Automatic Border Control systems (ABCs). Since the check-up subject is a document's photo and not a living person, active liveness detection is barely usable in this scenario.
Another concern surrounding face morphs states that a perpetrator can use a morphed photo to get identified and authorized as their victim, leading to a number of criminal schemes. Due to its rising criminal threat, facial morphing has been receiving attention from researchers and authorities. In 2018, a German activist managed to obtain a government-issued passport, which contained his photo morphed with a face of an Italian politician. Two years later, Germany officially banned "photo morphing in passports", forcing all applicants to either take photos at the passport office or use a secured submission network.
Among other issues is the easiness of producing a morphed image today. Currently, there are plenty of publicly available tools, which allow nearly spotless face blending: Photoshop, Face Swap Online, FaceMorpher, Magic Morph, and others. Facial morphing detection, has therefore emerged as a countermeasure to this threat, and allows detecting blended or morphed face images. Today's facial anti-spoofing with its types, countermeasures, and challenges, sees facial morphs as a viable threat.
Morphed Face Image Manual Detection
Experts note that spotting a face morph with a naked eye is challenging, if not impossible at all. After blending is complete, images regularly undergo post-processing, which is meant to erase all visible artifacts left by the facial morphing procedure: skin color inconsistencies, layering clues, warping, distortion, unnatural face toponymy, and so on.
The challenge of detecting morphed faces is much harder when it comes to identification documents. As noted in a study, face morphs for ID documents are mostly printed on small-sized pieces of photographic paper — the typical ID photo size is 3.5 x 4.5 cm or 1.3 x 1.7 inches. So, even if there are visible artifacts, a human observer will most likely fail at Morphed Attack Detection (MAD).
Nonetheless, experts have proposed a few techniques that may allow manual Morphed Attack Detection:
BRISQUE
Blind/Referenceless Image Spatial Quality Evaluator or BRISQUE proposes a default model for automatic evaluation of image quality. As a study reports, the model shows a high value for bona fide images, meanwhile morphs, even if uncompressed and of irreproachable quality, show a lower evaluation.
Artifact detection
Artifacts usually appear when a morphed image lacks precision in placement of landmarks, such as ears, head contour, iris, etc. As a result, a morphed image may include artifacts such as the so-called "ghost artifact". Shadow anomalies, blur, graininess, overly hard edges, unnatural color gradients, and other artifacts also indicate a morph.
Plausibility
When images of people from different ethnicity, age, or gender groups are morphed, the resulting image can have a certain "unnaturalness" and low plausibility. An example is a morphed image containing people who have a considerable age gap.
Training
A proposed method for better manual detection is that personnel should receive instruction and training regarding the MAD. A study proves that without prior knowledge of the morph scenarios, a human operator would not recognize 68% of morphs.
Databases on Facial Morphing Attack Detection
A number of facial datasets focusing on morphed image exist, with some public and most in the private domain.
Private
- The first known dataset was proposed along with the "magic passport" concept. It contains 14 morphs based on 8 bona fide images and created with the GIMP GAP method. This database is exclusively digital and is not available to a broad audience. The same team later introduced two more datasets: one made with Sqirlz and another with triangulation using the dlib landmark method.
- A bigger dataset was developed by Indian researchers. It was created with the GIMP GAP, adding the GNU image manipulation. It has a big sample collection of 450 images, featuring ethnic diversity. Later the same researchers proposed a dataset made with the OpenCV tool.
- A dataset developed by Makrushin et al., was made with complete and splicing morph methods. The former includes facial geometry of the source images, offering 1,326 samples. The latter clips out the "face pixels" out of the input image, offering 2,614 samples.
- Scherhag et al., developed the first dataset to feature physical (print/scan) morphs. Produced with GIMP GAP, it has 231 morph samples.
Public
- The first public dataset was developed by Biometrix. It is based on the NIST FERET database and offers 1,082 samples.
- FRLL-Morphs by IDAP is another publicly available morph dataset. It was created with WebMorpher, StyleGAN2, OpenCV and FaceMoprher, thus offering 4 types of sample material.
The number of facial morphing databases is still low compared to other technologies. As a result, the lack of training material greatly hinders the development of morph detection tools.
Facial Morphing Attack Detection
Morphed Attack Detection (MAD), used in morph antispoofing, includes four stages:
Data preparation
In this stage, the visual data — like a grayscale Trusted Live Capture (TLC) image — is normalized in terms of size, subject’s pose, face cropping, and other variables. This is necessary to ensure effective analysis. The dlib algorithm is often used for locating facial landmarks, ensuring reliable face liveness detection.
Feature extraction
Next, a feature vector is calculated. This is necessary for extracting spatial data and other vital parameters. For example, an image can be divided into cells for retrieving and connecting histograms. Besides, three types of descriptors can be used at this step:
- Texture
- Gradient-based
- Learned by a deep neural network
Texture descriptors can reveal artifacts left by morphing: landmark overlapping, iris artifacts, etc. Gradient-based descriptors help extract the Histograms of Oriented Gradients. Deep Convolutional Neural Networks allow extracting features that are vital for statistical analysis.
Feature preparation
The retrieved feature vectors need to be prepared to allow classifier training. Classifiers may require normalized data with multidimensional characteristics, and so on, depending on their type. Therefore, feature preparation is an essential step.
Classifier training
Finally, classifiers are trained based on the previously prepared data.
Face Demorphing
Face demorphing is an alternative method to MAD, and consists of two steps:
- A person’s live image is compared to the Electronic Machine Readable Travel Document (eMRTD).
- The suspected morph image featured in the ID is reverted to detect the owner’s real identity.
As a result, the system can automatically raise an alarm if attacked. The tests were orchestrated with the help of four SDKs successfully examined during the Face Recognition Vendor Test (FRVT). This method corresponds with the passive liveness detection doctrine.
Experiments & Evaluation of Methods
Various tests were held to attest MAD solutions. For example, one of the tests showed that error rate becomes much higher if low-quality, grayscale images are used for evaluation. (EER 17% if compared to EER 2.5% from the high-quality image test.)
Cross-dataset examination also shows poorer results, which indicates that MAD datasets contain highly specific parameters. At the same time, mixed-dataset testing shows better results with the error rate of just 35%.
It is suggested that ISO/IEC 30107-3 liveness certification standard with its APCER/BPCER rates can be used for evaluating MAD solutions. As a result, Face Recognition Vendor Test by NIST offers its own accuracy rating for morph detection based on these metrics.
FAQ
Face morphing — Definition
Face morphing is an attack type that merges multiple faces in order to fool face recognition systems.
Face morphing is a spoofing technique used to bypass facial recognition systems. In essence, face morphing ‘merges’ facial features of two or more individuals into one. As a result, a system like Automatic Border Control (ABC) can verify an illegal infiltrator as another legitimate person and let them pass.
Facial morphing is frequently used to counterfeit physical IDs. An additional challenge is that morphing is often hard to detect with a naked eye and morphed faces used in IDs are printed on small pieces of paper. Moreover, tools for morphing faces are freely available: Adobe Photoshop, Morph Thing, and others.
How to detect a morphed face manually?
Face morphing can be detected manually through various inconsistencies, however it is difficult and less accurate than automatic detection.
Face morphs can be detected manually using a few techniques:
- BRISQUE. Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) allows the user to promptly evaluate image quality. Morphed images in most cases get a low value.
- Plausibility check. Morphs created from images of two people with different ages, ethnicities, or even genders can evoke a doubt.
- Artifact detection. If a morph lacks finesse, it will have misplaced ears, nose, eyes (facial landmarks), inconsistent complexion, and ‘ghost artifacts’.
Ghost artifacts appear when one of the images used for a morph spoofing attack transparently overlaps the end result.
What are the main morphed image datasets?
A number of morph datasets exist but most are private and unavailable to the public.
To train and test antispoofing solutions, a number of morph datasets were developed. However, the majority of them are not publicly available. The first known morph dataset was connected to the research, which mentioned the ‘magic passport’ concept.
There are two main datasets available for the broad public. The first one is the Biometrix dataset: it’s based on the older FERET database and features 1,082 morphed samples. The second dataset is FRLL-Morphs developed by IDAP. It was produced with four tools: FaceMoprher, WebMorpher, StyleGAN2, and OpenCV, thus offering four types of spoof material.
How to detect face morphing attacks?
Face morphing detection involves manual and machine methods.
Spoof detection in regard to facial morphs can be both manual and automatic. It includes techniques such as Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE), ghost artifact and inconsistency detection, as well as plausibility check.
Compared to machine detection, manual detection is conducted with mixed success. Machine detection is far more reliable. It involves a specific pipeline with four steps: Data preparation, Feature extraction, Feature preparation and Classifier training.
Face demoprhing is an alternative technique based on the ID image reversion and real identity detection.
What morphed images are easier to detect?
Morphed images with low quality or bigger size can be detected with the naked eye.
Not all the facial morphs are produced with competence. Sometimes, artifacts left by image merging are quite visible and can reveal a spoofing attack. Among them are blurriness around the facial contour/hair, distortion, shadow anomalies, and misplaced facial landmarks: ears, nose, lips, philtrum, and so on.
Ghost artifact — when one of the donor images overlaps the photo in a ‘ghostly’, translucent manner — is a clear indication of a morphing attempt. Low plausibility of an image can also help spot a morph, especially when images of people from different ethnicities or age groups are blended.
Larger images allow easier detection manually as humans are able to spot any inconsistencies, however, morphing is mostly used on ID sized images where detection with naked eye is difficult.
Are there any databases of morphed faces?
A number of morphed image datasets exist with just a few being public.
Morphed image datasets have been introduced to mitigate morphing attacks. However, a large portion of those are private or licensable. The very first morphing dataset was offered within a survey, which highlighted a magic passport theory. It contains 14 morph images and remains private.
The first publicly available dataset for morph anti-spoofing was released by Biometrix. It contains 1,082 samples and is based on the FERET database. Another morph dataset available for usage is FRLL-Morphs by IDAP. It contains 4 image types made with StyleGAN2, OpenCV, FaceMoprher, and WebMorpher.
References
- Face morphing could catch criminals
- The magic passport
- Face morphing threat to biometric identity credentials’ trustworthiness a growing problem
- Germany bans digital doppelganger passport photos
- Face morphing tutorial in Adobe Photoshop
- Morph Thing allows anyone to easily morph two faces into one
- Morph Creation and Vulnerability of Face Recognition Systems to Morphing
- BRISQUE
- Face Recognition Systems Under Morphing Attacks: A Survey
- Fraudulent ID using face morphs: Experiments on human and automatic recognition
- Face Morphing Attack Generation & Detection: A Comprehensive Survey
- Facial landmarks with dlib, OpenCV, and Python
- The first public dataset
- NIST FERET database
- FRLL-Morphs is a dataset of morphed faces based on images selected from the publicly available Face Research London Lab dataset.
- Sample from Biometrix’s dataset
- Face Morphing Attack Detection Methods
- Face demorphing in the presence of facial appearance variations
- Face Recognition Vendor Test
- Practical Evaluation of Face Morphing Attack Detection Methods
- FRVT MORPH
