Face anti-spoofing to combat mask attacks: A remote photoplethysmography approach (P C Yuen et al.)

Research on face recognition technology over the past three decades has mainly focused on achieving high recognition accuracy of facial images that contain variations, for instance, in illumination, pose, scale and expression. Many algorithms with excellent recognition performance have been developed, and face biometric systems have consequently been deployed in many practical applications in the past decade. Although system accuracy is an important factor for practical face biometric systems, security is also a concern. The system has to be able to reject illegal users who present an imitation or fake face of an enrolled user, which is referred to as a face spoofing attack.

Due to the popularity of social networks, it is very easy to obtain biometric information on the face from the Internet. It is common for people to upload images/videos to share their daily activities with friends, and it is not difficult to obtain such images/videos from open sources. A face spoofing attack may use one of three approaches, namely displaying a printed photo, playing back a face video or wearing a 3D facial mask. Anti-spoofing algorithms that use the texture of the human face and multi-spectral analysis perform very well for printed photos and video replay attacks. Human computer interaction (HCI) methods, such as asking the user to blink his/her eyes or move his/her lips, have also been proposed for detecting live faces. It has recently become possible to fabricate low cost, hyper-real 3D colour masks off-the-shelf. As the colour and texture of such masks look real, they can be very difficult to distinguish from real faces. At the same time, simple HCI-based liveness detection may not work well. Experimental results have also shown that state-of-the-art face anti-spoofing methods are not good enough to detect a mask attack, and a masked face can successfully spoof face recognition systems. Therefore, there is an urgent need to develop more powerful face anti-spoofing algorithms to prevent mask attacks.

As even humans find it difficult to distinguish a hyper-real silicon masked face, image processing and analysis methods based on conventional face appearance may not be feasible. As such, this project proposes a new approach that tackles the problem in the most fundamental way, by analysing the blood volume flow in the living tissues of faces. To achieve this, we will use remote photoplethysmograph (rPPG) technology, an optical technique for detecting blood volume flow at a distance. The nature of rPPG means that it does not work on a masked (fake) face (please refer to Section 2.3 for details), which is a good property for this project. However, rPPG is sensitive to illumination changes and head motion, which are the two typical variations that occur when recognising faces from videos. As such, we need to develop a new rPPG method to handle illumination variations and head motion. To achieve this, we propose a new local rPPG approach with multiple local face regions. Adaptive filter and signal separation methods will be employed and investigated for the development of the new local rPPG model. Moreover, a set of ‘optimal’ local face regions which are favour for rPPG detection will be learnt.

This project will develop new rPPG methods and, therefore, new countermeasures for face antispoofing. The results of this project will also have great potential for application in the medical and healthcare fields, such as for heart rate monitoring during certain treatments in which traditional wired measurement is not plausible, and more comfortable long-term monitoring of vital signs in patients with chronic diseases.

Grant Support:

This project is supported by the Research Grants Council (RGC), Hong Kong SAR, China (Project HKBU12201215).

For further information on this research topic, please contact Prof. P C Yuen.