Siwei Lyu, University at Albany – Detecting DeepFake Videos

On University at Albany Week: To help detect a DeepFake video, looks at the eyes.

Siwei Lyu, associate professor of computer science, discusses the battle against DeepFakes.

Siwei Lyu is an associate professor in the Department of Computer Science, College of Engineering and Applied Sciences at the University at Albany, State University of New York. He is also the Director of Computer Vision and Machine Learning Lab (CVML) at UAlbany. Lyu’s research interests include digital image forensics, computer vision, computational neuroscience and machine learning.

Detecting DeepFake Videos

A new form of misinformation spread online early this year, called DeepFake vidoes, which use a type of machine-learning system known as a deep neural network, to synthesize images of a person’s face, to create fake videos looking very realistic. Because of the potential impact of misuse, detecting deepfake videos becomes a pressing need. 

Our recent work has suggested a reliable method to distinguish real videos from deepfakes. We observed that subects in deepfake videos blink a lot less frequently in comparison to real people. This is because deepfake algorithms are obtained using images of faces that are available on the internet. Even for people who are photographed often, few images are available online showing their eyes closed.  Not only are photos like that rare, but photographers don’t usually publish images where the main subjects’ eyes are shut.

Without training images of people blinking, deepfake algorithms are less likely to create faces that blink normally. This gives us an inspiration to detect DeepFake videos. We developed a method to detect when the person in the video blinks, and use the detected blinking rate to determine if the face is from a live person. This method achieves an overall 95 percent detection rate currently. 

This is not the final word on detecting deepfakes. The competition between generating and detecting fake videos is analogous to a chess game. In particular, blinking can be added to DeepFake videos by including face images with closed eyes or using video sequences for training. While forgers are getting better at making deceptive videos, we, along with others in the research community will continue to find ways to detect them.