Deepfake Detection Software: Types and Practical Application

Deepfake detection software is a specialized tool for detecting fake media, which can be available to regular users

Deepfake Detection: A Brief History

The rise of synthetic media has naturally caused a demand for reliable deepfake detection tools, some of which show rather impressive results.

Deepfake detection technology allows prompt recognition of a piece of fabricated media with high accuracy based on liveness signals. Distinguishing fake footage, photo or audio from authentic media is also necessary, as it helps to tackle the so-called liar’s dividend.

Deepfake detection had been an essential technology ever since deepfakes came into existence in the 1990s. However, when the first specimen appeared on Reddit platform in 2017, they caused increased concern. The falsified media were labeled a "calamity" and caused numerous concerns among the researchers, as well as among people who believed they could be potential deepfake targets.

Deepfake usage allegations sparked protests in Myanmar in 2021

Currently, a number of deepfake detection tools exists, ranging in sophistication, accuracy rates, and their accessibility to common users. One of the core ideas behind these tools is to provide a reliable detection technique to both experts and regular people so that deepfake threats such as mass-disinformation can be excluded.

Deepfake Detection Usage: Positive & Negative Aspects

A fake speech by US president Kennedy is the first known example of Deepfakes created using Video Rewrite tool — The first known deepfake software Video Rewrite was applied to a Kennedy’s speech

Broad availability of deepfake detection tools has raised opposing opinions in the expert community. A roundtable was held by Partnership on AI and nonprofit organization Witness to outline benefits and possible downsides of proliferating this technology.

The negative aspects include:

Hacking. While having first-hand access to a deepfake detection tool, fraudsters can learn to bypass and fool it. This can be achieved via reverse engineering, trial and error, code tampering, and other practices. In the past, a similar concern was raised in regard to passive liveness detection systems.
Limited availability. While proliferating a "home-use" anti-deepfake tool would be easy in the "Global North" — Europe, North America, and other regions — it would be hindered in less developed countries. Poor tech-literacy and limited access to the internet are among the main obstacles.
Trust issues. Regular users may discard detection results due to poor understanding of how the technology works. User’s overconfidence can also be a hurdle.
Lack of generalization. The deepfake detection solutions are trained on different datasets that contain videos and images from various sources: YouTube, Facebook, studio shots, surveillance cameras, and so on. Thus, a solution trained on a certain dataset may fail when facing data from a completely different source.
Privilege issues. If a deepfake detection tool becomes a privilege of the political authorities, it will automatically lose its legitimacy. Legal attempts at depriving journalists, fact-checkers, activists, as well as regular users of such technologies are also seen as a vital threat.
Insufficient infrastructure. The roundtable mentions an incident when the detector dubbed Deepfake-o-meter was made public for the first time. The number of requests was so big, it promptly crashed the servers.

Deepfake video of Gabon president Ali Bongo in 2018 — Gabon’s president Ali Bongo was in the center of a deepfake scandal in 2018

However, a number of positive aspects were also outlined:

Increased awareness. A simple and publicly available detector can prevent the spread of deepfakes and cheapfakes. In turn, this will hinder some unscrupulous practices: disinformation, slander, opinions manipulation, and so on.
Standardization. Cooperative efforts can lead to establishing universal guidelines, rules, and standards in media and for deepfake software vendors. This environment will make it much harder for deepfakes to appear and circulate.
Access for specialists. Journalists, whistleblowers, and fact-checkers can access more advanced digital forensic systems exclusively, which can help filter out malicious actors.

These positive aspects are part of the 6 key points to "responsible AI" and "information integrity" proposed by the roundtable.

A deepfake sample created by morphing faces of two women wot create a fake video — A deepfake sample created through face morphing (DFDC dataset)

Deepfake Software & Online Platforms for Deepfake Detection

In a response to the growing deepfake software and open sources like FaceSwap, a cohort of publicly available detectors were designed by various companies and startups.

Sensity

Sensity is an online platform dedicated to deepfake detection. It seeks to improve detection aspects such as remote KYC, face recognition, manual document check, as well as revealing media altered with deep learning and AI.

It allows uploading files in multiple formats — MP4, JPEG, TIFF — which then undergo a scanning procedure. According to the company, Sensity takes one second to reveal a forgery with an accuracy of 98.1%. Face swapping is mentioned as one of the targeted fraud techniques.

It is mentioned that the system can successfully tackle dating website scams, identity concealing, falsified IDs, and other types of fraud performed, among all else, with the help of the Generative Adversarial Networks (GANs).

Number of deepfakes created from 2019 to 2020 shows high increase — Spiking growth of deepfakes from 2019 to 2020

Deepware Scanner

Deepware Scanner is an open-source forensic tool. Deepware had been researching deepfakes since 2018, developing methods to detect them. Deepware Scanner is unique as it is deliberately tested on multiple data sources, including organic and live videos.

The scanner is based on EfficientNet-B7 — a model of the convolution neural network architecture, which is based on uniform scaling of all CNN dimensions. In turn, this provides higher accuracy and general cost-efficiency.

Its primary training dataset is the CFDC dataset, which contains 120,000 consented videos. The test datasets included 4chan Real, MrDeepFakes, Celeb-DF YouTube, and others.

Accuracy results of Deepware scanner test using different datasets — Deepware scanner test results

Authors behind Deepware Scanner state that community support is inescapable in tackling deepfakes. Hence, the project remains open-source and welcomes researchers.

Deepware dataset with images of twins used for antispoofing detection — Deepware has its own datasets featuring twins

Microsoft Video Authenticator

The deepfake detection tool dubbed Microsoft Video Authenticator was developed as part of the Microsoft Defending Democracy Program, which strives to prevent the spread of disinformation.

It is capable of analyzing both static images and videos in real-time mode. In the case of the video materials, it can spot blending boundary and subtle fading or grayscale elements. These are the 'clues' typically left by a deepfake and virtually invisible to the naked eye.

Video Authenticator was trained on Face Forensic++ and tested on the CFDC dataset. Test results and accuracy of the solution, however, were not disclosed. Moreover, Microsoft mentions that an effective technology would not be sufficient alone, and proposed several auxiliary initiatives such as NewsGuard.

Video Authenticator used to detect originality of image of woman — Footage demonstrating Video Authenticator at work

DeepDetector & DeepfakeProof from DuckDuckGoose

DeepDetector and DeepfakeProof are designed for detecting digital forgeries. While the former is meant for the corporate domain, the latter is a free-to-use browser plugin for everyone.

Deepfake Detector has its own API and is available in two iterations: an online platform and a standalone application. Meanwhile, DeepfakeProof is installed as a browser extension. It scans every website visited by a user, alerting them if any fabricated media is present.

Deepfake-o-Meter

Deepfake-o-meter is an online deepfake detection platform, which allows:

Analyzing suspicious video files.
Wrapping individual algorithms for running them on various servers.
Comparing various algorithms and their effectiveness on a single input.

A video can be uploaded via a URL link or as a file with a maximum size of 50 MB. To detect fake media, it employs Xception, ClassNSeg, EfficientNet-B3, CNNDetection, spatial pyramid pooling, mesoscopic image properties analysis, and other techniques.

Front and back-end architecture of Deepfake-o-meter detector — Deepfake-o-meter architecture

Reality Defender

Reality Defender is a deepfake detection utility designed by the AI Foundation. It is meant for both commercial usage as a fraud prevention tool and for noncommercial purposes such as disinformation tackling, investigations, and so on. Initially, Reality Defender was conceived as a non-partisan tool to maintain ethical and truth standards. (Especially during presidential elections).

WeVerify

WeVerify is a browser extension that can recognize false media and information. It’s equipped with such know-hows as image context analysis, cross-network search, Optical Character Recognition (OCR), integration of Database of Known Fakes (DBKF) for content credibility matching, and so on. It has received the US Paris first prize from the Global Engagement Center of the US State Department and DFRLab.

Intel's FakeCatcher

FakeCatcher is a deepfake detector, possibly based on the photoplethysmographic analysis, as it is proclaimed “the first deepfake detection algorithm that uses heart rates”. This type of analysis pays attention to how skin coloring of a subject present in a video changes in accordance with their breathing patterns. While natural to humans, this phenomenon is never observed in AI-fabricated videos.

FAQ

What are the ways of detecting deepfakes?

Deepfake detection involves understanding of complex techniques of video/audio matching.

Although deepfakes have evolved to become highly realistic, there are a few effective ways to detect them.

One method to spot deepfakes is based on data modalities and includes:

Audio spectrogram analysis
Video spatio-temporal features
Audio-video inconsistency analysis

For example phoneme vs. viseme comparison can be used for unmasking synthesized media or a video, in which a different audio track is used.

Another promising method involves heart rate estimation. A person’s face color shows subtle changes in response to oxygen levels in the surrounding. This change, while not detectable by a naked human eye, can be spotted using special detectors that can be used in conjunction with recognition systems to detect deepfakes.

Is there any software for detecting deepfakes?

A number of deepfake detection softwares are available online.

As the emergence of deepfakes has caused considerable concern, companies have presented their own antispoofing software that can detect deepfakes quickly and accurately, while staying available to the broad audience.

Sensity is an earlier detection solution that is available online and allows scanning files in multiple formats. Deepware Scanner is an open source-tool based on the EfficientNet-B7 model. DeepfakeProof is a free browser extension that scans webpages for fake media. Microsoft proposed its own antispoofing app dubbed MVA, however, it is not yet available to the general public.

How to detect deepfakes online?

Deepfakes can be detected with the help of specialized applications and platforms.

A number of antispoofing software tools have been proposed, as well as some other solutions. Some applications, like DeepfakeProof, autonomously scan webpages to detect potential fake media. Others, like Sensity, allow uploading files in its scanner for authenticity checks. Other similar solutions are Deepfake-o-Meter, Reality Defender, etc.

Adobe and New York Times in collaboration with a few other organizations proposed Content Authenticity Initiative (CAI) as a tool for deepfake and antispoof checking. The core idea is that CAI will allow content authors to protect their creations via cryptography. In turn, this will help to track down the origins of any media file.