Catch up on the latest AI articles

Webface-OCC, A New Benchmark For Occlusion Face Recognition

Webface-OCC, A New Benchmark For Occlusion Face Recognition

Face Recognition

3 main points.
✔️ We propose a public dataset for occlusion-aware face recognition
✔️ Compared with the previous
Introduces a mapping method for occlusion that is more suitable for practical conditions
✔️ Re-trained. ArcFace significantly improves the recognition accuracy when wearing a mask, with almost no loss in normal face recognition accuracy

When Face Recognition Meets Occlusion: A New Benchmark
written by Baojin Huang, Zhongyuan Wang, Guangcheng Wang, Kui Jiang, Kangli Zeng, Zhen Han, Xin Tian, Yuhong Yang
(Submitted on 4 Mar 2021)
Comments: Accepted to ICASSP 2021.

Subjects: Computer Vision and Pattern Recognition (cs.CV) 



Since the first year of facial recognition in 2019, facial recognition technology has been introduced everywhere. However, from 2020 onwards, masks will be commonly worn as a way to prevent the spread of coronavirus (COVID-19) worldwide.

It is known that the recognition accuracy of existing face recognition models is significantly degraded under occlusion conditions, such as when masks are worn. One of the main reasons for this is the lack of datasets that take occlusion into account.

Currently, there is no public dataset for face recognition that takes occlusion into account. Although there have been some studies on occlusion-aware face recognition models including masks and sunglasses, all of them have constructed their own datasets.

However, all of them have constructed their own datasets, and the datasets constructed by them are not sufficient because they are very different from practical conditions.

For example, (a) is a sample of the dataset used in MaskNet reported in 2016. In this dataset, we use images with randomly applied black boxes of multiple sizes. The only occlusion type is black boxes, which is expected to lower the generalization performance of the model. Also, the occlusion is unnatural considering the practical conditions.

(b) is a sample of three used datasets in the Pairwise Differential Siamese Network (PDSN) reported in 2019. Here, three occlusion types are available. The number of occlusion types has increased compared to (a), but the position and size of the occlusion are still unnatural considering the practical conditions.

(c) is a sample of the dataset used in the wID reported in 2020. Here, square boxes are randomly applied to face images. The synthesis method is very simple and still not adapted to practical conditions.

Other methods using GANs, which have attracted much attention in recent years, can produce visually more natural occlusion images, but However, the detailed information is changed, and the face recognition models trained on these images often do not work well under practical conditions.

Thus, although there are some existing occlusion-aware datasets, most of them are very different from practical conditions.

In this paper, we provide a new public occlusion-aware dataset, Webface-OCC, to improve this situation.

(d) is the sample data of Webface-OCC. It consists of 10,575 subjects and 804,704 face images, including various occlusion types.

What is "Webface-OCC"?

Webface-OCC is built based on CASIA-Webface, which is a commonly used dataset for face recognition. CASIA-Webface contains data with slight occlusions, and models trained on CASIA-Webface have shown high performance in face recognition with small occlusions.

By enhancing this occlusion, Webface-OCC is reconstructed as a new dataset that is useful for improving face recognition performance for occlusions.

The figure below shows a sample of Webface-OCC. Instead of masking faces randomly with square boxes as before, we have data masking faces with masks and sunglasses, which are often faced in practical conditions. We also include different textures, colors and sizes: the first line is a normal face image and the second/third line is an occluded face image.


Webface-OCC first provides several types of (a) textures/color and (b) masks/sunglasses as shown in the figure below. Next, we obtain facial feature points from an unmasked normal face image.

Finally, using the facial feature points, we generate an image with occlusion by mapping the mask to a position that covers the mouth and nose area, and the sunglasses to cover the eye area, while adjusting the angle and size.

We increase the data size by applying a combination of multiple occlusion types. The final dataset contains 10,575 subjects and 804,704 face images.

In addition, the dataset contains both normal and occlusion face images for each ID, and the ratio is equal (see the figure below).


We evaluate the model trained by Webface-OCC in two cases.

One is the case using Labeled Faces in the Wild (LFW), Celebrities in Frontal-Profile in the Wild (CFP-FP), and AgeDB-30, which are used for general face recognition, and The other case uses the recently proposed masked face recognition datasets LFW-mask, CFP-FP-mask, AgeDB-30-mask, and RMFRD (Real-World Masked Face Dataset). FW-mask, CFP-FP-mask, and AgeDB-30-mask are data to which masks are pseudo-applied to the face images of LFW, CFP-FP, and AgeDB-30, and the data size and so on are the same.

LFW-mask, CFP-FP-mask, and AgeDB-30-mask datasets are the result of adding masks to the original datasets, and the size and scale of the data are not changed.

The models are based on six representative face recognition models CenterFace, SphereFace, FaceNet, CosFace, ArcFace and MaskNet. Of these, FaceNet and ArcFace have also been validated with models retrained on the WiderFace dataset.

The results are shown in the table below, although in the case of CFP-FP and AgeDB-30 the accuracy is much lower than LFW due to the effects of face orientation and age differences. The accuracy of the model trained on Webface-OCC is only about 1% lower than that of the original model, indicating an overall high performance on general face recognition datasets.

In addition, the re-trained models (FaceNet and ArcFace) significantly outperform the original models. For example, ArcFace is 36.22%, 29.14%, 27.04%, and 15.03% more accurate than the original model on four masked face recognition datasets (LFW-mask, CFP-FP-mask, AgeDB-30-mask, and RMFRD), respectively .

In other words, the retrained model succeeds in significantly improving only the performance on the occlusion face recognition dataset, while retaining the original impact on the general face recognition dataset. At the same time, the recognition accuracy is lower on the real occlusion face recognition dataset (RMFRD) compared to the simulated occlusion face recognition datasets (LFW-mask, CFP-FP-mask, and AgeDB-30-mask).

This may be due to the inability to accurately recognize unknown occlusions in RMFRD, or the fact that the subjects are public figures and are intentionally disguised so that their identities are not revealed.


This paper proposes a public dataset for occlusion-aware face recognition. Unlike conventional methods, we propose a method to synthesize occlusions that is closer to practical conditions by mapping facial feature points. We propose a method to synthesize occlusions that is closer to practical conditions. By applying this method to the existing Webface dataset, we construct a public dataset with large-scale occlusions.

Furthermore, ArcFace re-trained on this dataset has a LFW-mask and RMFRD datasets, achieving high accuracy of 97.08% and 78.25%, respectively.

It has been reported by many international organizations, such as NIST, that the accuracy of conventional face recognition models significantly decreases with the wearing of masks. It is expected that this Webface-OCC will lead to larger, more diverse, and more accurate occlusion face recognition datasets and improved accuracy of face recognition models in the future.


If you have any suggestions for improvement of the content of the article,
please contact the AI-SCHOLAR editorial team through the contact form.

Contact Us