Noise-attention-based forgery face detection method

Bolin ZHANG; Chuntao ZHU; Qilin YIN; Jingqiao FU; Lingyi LIU; Jiarui LIU; Hongmei LIU; Wei LU

doi:10.11959/j.issn.2096-109x.2023061

您当前的位置：

首页 >

文章列表页 >

Noise-attention-based forgery face detection method

Papers | 更新时间：2024-06-03

- Noise-attention-based forgery face detection method
- Chinese Journal of Network and Information Security Vol. 9, Issue 4, Pages: 155-165(2023)
- 作者机构：
  
  1. 中山大学计算机学院，广东广州 510006
  2. 广东省信息安全技术重点实验室，广东广州 510006
  3. 机器智能与先进计算教育部重点实验室，广东广州 510006
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2023061
  CLC： TP393
- Online First：2023-08，
  
  Published：25 August 2023
- 稿件说明：
移动端阅览
Bolin ZHANG, Chuntao ZHU, Qilin YIN, et al. Noise-attention-based forgery face detection method[J]. Chinese Journal of Network and Information Security, 2023, 9(4): 155-165.
DOI：

Bolin ZHANG, Chuntao ZHU, Qilin YIN, et al. Noise-attention-based forgery face detection method[J]. Chinese Journal of Network and Information Security, 2023, 9(4): 155-165. DOI： 10.11959/j.issn.2096-109x.2023061.

摘要

随着人工智能和深度神经网络的不断发展，图像生成与编辑变得越来越容易，恶意运用图像生成工具进行篡改伪造的现象层出不穷，这对多媒体安全以及社会稳定造成了极大威胁，因此研究伪造人脸的检测方法至关重要。人脸篡改伪造的方式和工具多种多样，在篡改的过程中可能留下不同程度的篡改痕迹，而这在图像噪声中都有一定程度上的反映。从图像噪声的角度出发，通过噪声去除的方式挖掘反映伪造人脸篡改痕迹的噪声成分，进一步生成噪声注意力，指导主干网络进行伪造人脸检测。使用 SRM 滤波监督噪声去除模块的训练，并将噪声去除模块所得到的噪声再次加入真实人脸图像中，形成一对有监督的训练样本，通过自监督的方式对噪声去除模块进行加强指导，实验结果说明噪声去除模块得到的噪声特征具有较好的区分度。在多个公开数据集上进行了实验，所提方法在 Celeb-DF 数据集上达到 98.32%的准确率，在FaceForensics++数据集上达到 94%以上的准确率，在 DFDC 数据集上达到 92.61%的准确率，证明了所提方法的有效性。

Abstract

With the advancement of artificial intelligence and deep neural networks

the ease of image generation and editing has increased significantly.Consequently

the occurrence of malicious tampering and forgery using image generation tools is on the rise

posing a significant threat to multimedia security and social stability.Therefore

it is crucial to research detection methods for forged faces.Face tampering and forgery can occur through various means and tools

leaving different levels of forgery traces during the tampering process.These traces can be partly reflected in the image noise.From the perspective of image noise

the noise components reflecting tampering traces of forged faces were extracted through a noise removal module.Furthermore

noise attention was generated to guide the backbone network in the detection of forged faces.The training of the noise removal module was supervised using SRM filters.In order to strengthen the guidance of the noise removal module

the noise obtained by the noise removal module was added back to the real face image

forming a pair of supervised training samples in a self-supervised manner.The experimental results illustrate that the noise features obtained by the noise removal module have a good degree of discrimination.Experiments were also conducted on several public datasets

and the proposed method achieves an accuracy of 98.32% on the Celeb-DF dataset

92.61% on the DFDC dataset

and more than 94% on the FaceForensics++ dataset

thus proving the effectiveness of the proposed method.

关键词

Keywords

references

XU W , KESHMIRI S , WANG G . Adversarially approximated auto encoder for image generation and manipulation [J ] . IEEE Transactions on Multimedia , 2019 , 21 ( 9 ): 2387 - 2396 .

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial nets [C ] // Advances in neural Information Processing Systems . 2014 :27.

LIU M , DING Y , XIA M , et al . Stgan:a unified selective transfer network for arbitrary image attribute editing [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2019 : 3673 - 3682 .

CHOI Y , CHOI M , KIM M , et al . Stargan:unified generative adversarial networks for multi-domain image-to-image translation [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 : 8789 - 8797 .

KARRAS T , LAINE S , AITTALA M , et al . Analyzing and improving the image quality of stylegan [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2020 : 8110 - 8119 .

王鹏程 , 郑海斌 , 邹健飞 , 等 . 面向商用活体检测平台的鲁棒性评估 [J ] . 网络与信息安全学报 , 2022 , 8 ( 1 ): 180 - 189 .

WANG P C , ZHENG H B , ZOU J F , et al . Robustness evaluation of commercial liveness detection platform [J ] . Chinese Journal of Network and Information Security , 2022 , 8 ( 1 ): 180 - 189 .

WAHEED A , GOYAL M , GUPTA D , et al . An optimized dense convolutional neural network model for disease recognition and classification in corn leaf [J ] . Computers and Electronics in Agriculture , 2020 ,175:105456.

CHOLLET F . Xception:deep learning with depthwise separable convolutions [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2017 : 1251 - 1258 .

MATERN F , RIESS C , WINTER M S 2019 I , et al . Exploiting visual artifacts to expose deepfakes and face manipulations [C ] // 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) . 2019 : 83 - 92 .

GÜERA D , DELP E J . Deepfake video detection using recurrent neural networks [C ] // 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) . 2018 : 1 - 6 .

NIRKIN Y , WOLF L , KELLER Y , et al . Deepfake detection based on the discrepancy between the face and its context [J ] . arXiv Preprint arXiv:2008.12262 , 2020 .

MCCLOSKEY S , ALBRIGHT M , ACST H . Detecting GAN-generated imagery using color cues [J ] . arXiv Preprint arXiv:1812.08247 , 2018 .

PENG B , WANG W , DONG J , et al . Optimized 3D lighting environment estimation for image forgery detection [J ] . IEEE Transactions on Information Forensics and Security , 2016 , 12 ( 2 ): 479 - 494 .

YU N , DAVIS L S , FRITZ M . Attributing fake images to GANs:Learning and analyzing GAN fingerprints [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . 2019 : 7556 - 7566 .

FRANK J , EISENHOFER T , SCHÖNHERR L , et al . Leveraging frequency analysis for deep fake image recognition [C ] // International Conference on Machine Learning . 2020 : 3247 - 3258 .

QIAN Y , YIN G , SHENG L , et al . Thinking in frequency:face forgery detection by mining frequency-aware clues [C ] // European Conference on Computer Vision . 2020 : 86 - 103 .

FRIDRICH J , KODOVSKY J . Rich models for steganalysis of digital images [J ] . IEEE Transactions on Information Forensics and Security , 2012 , 7 ( 3 ): 868 - 882 .

LI Y , YANG X , SUN P , et al . Celeb-df:a large-scale challenging dataset for deepfake forensics [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2020 : 3207 - 3216 .

RÖSSLER A , COZZOLINO D , VERDOLIVA L , et al . Faceforensics++:learning to detect manipulated facial images [C ] // Interna- tional Conference on Computer Vision(ICCV) . 2019 : 1 - 11 .

AGARWAL S , FARID H , EL-GAALY T , et al . Detecting deepfake videos from appearance and behavior [C ] // 2020 IEEE International Workshop on Information Forensics and Security(WIFS) . 2020 : 1 - 6 .

NIRKIN Y , WOLF L , KELLER Y , et al . Deepfake detection based on discrepancies between faces and their context [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 ,( 8 ).

MCCLOSKEY S , ALBRIGHT M . Detecting GAN-generated imagery using saturation cues [C ] // 2019 IEEE International Conference on Image Processing (ICIP) . 2019 : 4584 - 4588 .

MARRA F , GRAGNANIELLO D , VERDOLIVA L , et al . Do GANs leave artificial fingerprints? [C ] // 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) . 2019 : 506 - 511 .

AGARWAL S , FARID H , Fried O , et al . Detecting deepfake videos from phoneme-viseme mismatches [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . 2020 : 660 - 661 .

BHATTACHARYA U , MITTAL T , CHANDRA R , et al . Step:spatial temporal graph convolutional networks for emotion perception from gaits [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . 2020 : 1342 - 1350 .

LI Y , CHANG M C , LYU S . In ICTU oculi:exposing AI created fake videos by detecting eye blinking [C ] // 2018 IEEE International Workshop on Information Forensics and Security (WIFS) . 2018 : 1 - 7 .

LI Y , LYU S . Exposing deepfake videos by detecting face warping artifacts [J ] . arXiv Preprint arXiv:1811.00656 , 2018 .

MATERN F , RIESS C , STAMMINGER M . Exploiting visual artifacts to expose deepfakes and face manipulations [C ] // 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) . 2019 : 83 - 92 .

QI H , GUO Q , JUEFEI X F , et al . Deeprhythm:exposing deepfakes with attentional visual heartbeat rhythms [C ] // Proceedings of the 28th ACM International Conference on Multimedia . 2020 : 4318 - 4327 .

KOOPMAN M , RODRIGUEZ A M , GERADTS Z . Detection of deepfake video manipulation [C ] // The 20th Irish Machine Vision and Image Processing Conference (IMVIP) . 2018 : 133 - 136 .

LI H , LI B , TAN S , et al . Identification of deep network generated images using disparities in color components [J ] . Signal Processing , 2020 ( 174 ): 107616 .

DO N T , NA I S , KIM S H . Forensics face detection from GANs using convolutional neural network [C ] // ISITC . 2018 : 376 - 379 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [J ] . arXiv Preprint arXiv:1409.1556 , 2014 .

TARIQ S , LEE S , KIM H , et al . Detecting both machine and human created fake face images in the wild [C ] // Proceedings of the 2nd International Workshop on Multimedia Privacy and Security . 2018 : 81 - 87 .

HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 770 - 778 .

NGUYEN H , YAMAGISHI J , ECHIZEN I . Capsule-forensics:Using capsule networks to detect forged images and videos [C ] // 2019 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP) . 2019 : 2307 - 2311 .

SABIR E , CHENG J , JAISWAL A , et al . Recurrent convolutional strategies for face manipulation detection in videos [J ] . Interfaces (GUI) , 2019 , 3 ( 1 ): 80 - 87 .

HSU C C , ZHUANG Y X , LEE C Y . Deepfake image detection based on pairwise learning [J ] . Applied Sciences , 2020 , 10 ( 1 ): 370 .

GANDHI A , JAIN S . Adversarial perturbations fool deepfake detectors [C ] // 2020 International Joint Conference on Neural Networks (IJCNN) . 2020 : 1 - 8 .

龚晓娟 , 黄添强 , 翁彬 , 等 . 基于双层注意力的 Deepfake 换脸检测 [J ] . 网络与信息安全学报 , 2021 , 7 ( 2 ): 151 - 160 .

GONG X J , HUANG T Q , WENG B , et al . Deepfake swapped face detection based on double attention [J ] . Chinese Journal of Network and Information Security , 2021 , 7 ( 2 ): 151 - 160 .

WU X , XIE Z , GAO Y , et al . Sstnet:detecting manipulated faces through spatial,steganalysis and temporal features [C ] // 2020 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP) . 2020 : 2952 - 2956 .

LIU Z , QI X , TORR P H S . Global texture enhancement for fake face detection in the wild [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2020 : 8060 - 8069 .

RONNEBERGER O , FISCHER P , BROX T . U-net:convolutional networks for biomedical image segmentation [C ] // International Conference on Medical Image Computing and Computer-Assisted Intervention . 2015 : 234 - 241 .

WANG F , CHENG J , LIU W , et al . Additive margin softmax for face verification [J ] . IEEE Signal Processing Letters , 2018 , 25 ( 7 ): 926 - 930 .

KING D E . Dlibml:a machine learning toolkit [J ] . The Journal of Machine Learning Research , 2009 , 10 : 1755 - 1758 .

BAYAR B , MEMBER S , STAMM M C . Constrained convolutional neural networks:a new approach towards general purpose image manipulation detection [J ] . IEEE Transactions on Information Forensics and Security , 2018 , 13 ( 11 ): 2691 - 2706 .

AFCHAR D , NOZICK V , YAMAGISHI J , et al . MesoNet:a compact facial video forgery detection network [C ] // 2018 IEEE International Workshop on Information Forensics and Security(WIFS) . 2018 : 1 - 7 .

TAN M , LE Q . Efficientnet:rethinking model scaling for convolutional neural networks [C ] // International Conference on Machine Learning . 2019 : 6105 - 6114 .

LIU H , LI X , ZHOU W , et al . Spatial-phase shallow learning:rethinking face forgery detection in frequency domain [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2021 : 772 - 781 .

TRAN D , BOURDEV L , FERGUS R , et al . Learning spatiotemporal features with 3D convolutional networks [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2015 : 4489 - 4497 .

CARREIRA J , ZISSERMAN A , COM Z , et al . Quo vadis,action recognition? a new model and the kinetics dataset [C ] // proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2017 : 6299 - 6308 .

LUO Y , ZHANG Y , YAN J , et al . Generalizing face forgery detection with high-frequency features [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2021 : 16317 - 16326 .

DOLHANSKY B , HOWES R , PFLAUM B , et al . The deepfake detection challenge (DFDC) preview dataset [J ] . arXiv Preprint arXiv:1910.08854 , 2019 .

WANG C , DENG W . Representative forgery mining for fake face detection [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2021 : 14923 - 14932 .

ZHAO H , ZHOU W , CHEN D , et al . Multi-attentional deepfake detection [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2021 : 2185 - 2194 .

Views

110

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Multi-source forgery detection method based on the fusion of color space features and frequency domain features

Cross-stream attention enhanced central difference convolutional network for CG image detection

Phishing detection algorithm based on attention and feature fusion

Deep learning vulnerability detection method based on optimized inter-procedural semantics of programs

Forgery face detection method based on multi-domain temporal features mining

Related Author

Wang Jiabao

Li Yang

Zhang Qi

Wang Jun

Miao Zhuang

Hou Yifan

HUANG Jinkun

HUANG Yuanhang

Related Institution

Institute of Command-and-Control Engineering, Army Engineering University of PLA

Guangdong Key Laboratory of Information Security Technology

China Internet Network Information Center

Services Computing Technology and System Lab, National Engineering Research Center for Big Data Technology

Hubei Key Laboratory of Distributed System Security

⁰