The Face Generated By GAN Can Be Judged By The Pupil
3 main points
✔️ High-quality StyleGAN-generated faces are found to be different from real human pupils
✔️ We propose a simple and effective method that can detect GAN-generated faces using irregular pupil shapes as cues
✔️ It is a good cue for humans to visually identify GAN-generated faces.
Eyes Tell All: Irregular Pupil Shapes Reveal GAN-generated Faces
written by Hui Guo, Shu Hu, Xin Wang, Ming-Ching Chang, Siwei Lyu
(Submitted on 1 Sep 2021)
Comments: Published on arxiv.
Subjects: Computer Vision and Pattern Recognition (cs.CV)
code：
The images used in this article are from the paper or created based on it.
first of all
We all know that with the development of GAN, it has become possible to generate face images that are indistinguishable from each other. In the midst of these technological developments, there are some bad sides, such as the appearance of fake social media accounts that use the generated faces as profile images. Therefore, with the development of GANs, approaches to detect generated face images using Deep Learning models have been developed. However, these approaches suffer from two serious challenges: lack of interpretability and low generalization among different generation methods due to the overfitting problem. Improvements have been made, for example, by identifying deficiencies in the GAN generative model in representing the interaction between human faces and the physical world and detecting the products. As an example, it has been proposed to use the discrepancy of corneal specular highlights between two generated eyes. However, the proposed method cannot be used in this way under a strict portrait setting where the light source and reflector are visible to both eyes and the eyes are far from the light source and reflector. To overcome these limitations and explore a more robust model, we propose a new physical method based on pupil shape.
technique
The method is simple. In order to detect the pupil shape artifacts as shown in the figure below and determine the face generated by the GAN, the method first automatically extracts the pupil masks of both eyes and evaluates whether they are elliptical or not.
For this purpose, the pupil cropping method is included as a method, but I will omit most of the methods this time.
experiment
data set
We use the FlickrFaces-HQ (FFHQ) dataset for images of real human faces, and GAN-generated human faces created by StyleGAN22. We have 1000 images of 1024×1024 resolution for each class.
result
An example of the analysis result of a real human face and the pupil of a human face generated by GAN is shown in the figure below.
The eyes of real people have a strong oval shape, which is highly reflected in their eyes.
BIoU Score
It is higher between the predicted pupil mask and the oval-fitted pupil mask. However, irregularly shaped pupil artifacts significantly reduce the BIoU score. The distribution of BIoU scores for the pupils of real and GAN-generated faces is also shown in the figure below. The two distributions are clearly separated, indicating that the irregular pupil shape is an effective means of distinguishing between real and GAN-generated faces.
restriction
This method also has some limitations: it uses the irregular geometry shape of the pupil to detect GAN-generated faces. This does not work well if there are diseases or other problems in the pupil or iris region. In addition, failures in pupil occlusion or pupil segmentation can also lead to incorrect predictions. In other words, it is assumed that the pupil is cleanly captured.
summary
We propose that a GAN-generated face can be judged by the irregular shape of its pupils. In the future, we will consider other types of inconsistencies between the two pupils of a GAN-generated face, such as different geometric shapes and the relative positions of the pupils of the two eyes. There are still some geometric features that are useful to determine, and we believe that a more convenient way to detect them will be proposed.
With the development of GAN technology, I believe that images that are difficult for people to distinguish will be generated. Considering the misuse of this technology, we will surely need this kind of technology, so we will keep our eyes on it.
Categories related to this article
