Deepfake swapped face detection based on double attention

Xiaojuan GONG; Tianqiang HUANG; Bin WENG; Feng YE; Chao XU; Lijun YOU

doi:10.11959/j.issn.2096-109x.2021032

您当前的位置：

首页 >

文章列表页 >

Deepfake swapped face detection based on double attention

Papers | 更新时间：2024-06-03

- Deepfake swapped face detection based on double attention
- Chinese Journal of Network and Information Security Vol. 7, Issue 2, Pages: 151-160(2021)
- 作者机构：
  
  1. 福建师范大学数学与信息学院，福建福州 350117
  2. 数字福建大数据安全技术研究所，福建福州 350117
  3. 福建省灾害天气重点实验室，福建福州 350001
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2021032
  CLC： TP393
- Online First：2021-04，
  
  Published：15 April 2021
- 稿件说明：
移动端阅览
Xiaojuan GONG, Tianqiang HUANG, Bin WENG, et al. Deepfake swapped face detection based on double attention[J]. Chinese Journal of Network and Information Security, 2021, 7(2): 151-160.
DOI：

Xiaojuan GONG, Tianqiang HUANG, Bin WENG, et al. Deepfake swapped face detection based on double attention[J]. Chinese Journal of Network and Information Security, 2021, 7(2): 151-160. DOI： 10.11959/j.issn.2096-109x.2021032.

摘要

针对现有Deepfake检测算法中普遍存在的准确率低、可解释性差等问题，提出融合双层注意力的神经网络模型，该模型利用通道注意力捕获假脸的异常特征，并结合空间注意力聚焦异常特征的位置，充分学习假脸异常部分的上下文语义信息，从而提升换脸检测的有效性和准确性。并以热力图的形式有效地展示了真假脸的决策区域，使换脸检测模型具备一定程度的解释性。在 FaceForensics++开源数据集上的实验表明，所提方法的检测精度优于MesoInception、Capsule-Forensics和XceptionNet检测方法。

Abstract

In view of the existing Deepfake detection algorithms

such problems as low accuracy and poor interpretability are common.A neural network model combining the double attention was proposed

which used channel attention to capture the abnormal features of false faces and combined the location of spatial attention to focus the abnormal features.To fully learn the contextual semantic information of the abnormal part of the false face

so as to improve the effectiveness and accuracy of face changing detection.In addition

the decision-making area of real and fake faces was shown effectively in the form of thermal diagram

which provided a certain degree of explanation for the face exchange detection model.Experiments on FaceForensics ++ open source data set show that the detection accuracy of proposed method is superior to MesoInception

Capsule-Forensics and XceptionNet.

关键词

Keywords

references

THIES J , ZOLLHOFER M , STAMMINGER M , et al . Demo of Face2Face:real-time face capture and reenactment of RGB videos [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2016 ( 1 ): 2387 - 2395 .

RÖSSLER A , COZZOLINOD , VERDOLIVAL , , et al . FaceForensics++:learning to detect manipulated facial images [C ] // International Conference on Computer Vision(ICCV) . 2019 : 1 - 11 .

LI Y , CHANG M , FARID H , et al . In ictu.oculi:exposing AI generated fake face videos by detecting eye blinking [C ] // International Workshop on Information Forensics and Security(WIFS) . 2018 : 1 - 7 .

COZZOLINO D , VERDOLIVA L . Noiseprint:a CNN-based camera model fingerprint [J ] . IEEE Transactions on Information Forensics and Security , 2020 ( 15 ): 144 - 159 .

DONG H , LIU F , STEHOUWER J , et al . On the detection of digital face manipulation [J ] . arXiv:1910.01717[astro-ph.CO] .

ZHOU C , WU M , LAM S . SSA-CNN:semantic self-attention CNN for pedestrian detection [J ] . arXiv:1902.09080v1[astro-ph.CO] .

NGUYEN H , YAMAGISHI J , ECHIZEN I . Use of a capsule network to detect fake images and videos [J ] . arXiv:1910.12467v2[astro-ph.CO] .

SELVARAJU M , COGSWELL M , DAS A . Grad-CAM:visual explanations from deep networks via gradient-based localization [C ] // International Conference on Computer Vision (ICCV) . 2017 : 618 - 626 .

YANG X , LI Y , LYU S . Exposing deep fakes using inconsistent head poses [C ] // 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP) . 2019 : 8261 - 8265 .

MATERN F , RIESS C , STAMMINGER M . Exploiting visual artifacts to expose Deepfakes and face manipulations [C ] // 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) . 2019 : 83 - 92 .

NATARAJ L , MOHAMMED T M , MANJUNATH B S , et al . Detecting GAN generated fake images using co-occurrence matrices [C ] // Media Watermarking,Security,and Forensics . 2019 .

LI Y , LYU S . Exposing Deepfake videos by detecting face warping artifacts [C ] // 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2019 : 46 - 52 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [C ] // 3rd International Conference on Learning Representations (ICLR) . 2015 .

HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . 2016 : 770 - 778 .

GUERA D , DELP E . Deepfake video detection using recurrent neural networks [C ] // 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) . 2018 : 1 - 6 .

DONAHUE J , HENDRICKS L , GUADARRAMA S , et al . Long-term recurrent convolutional networks for visual recognition and description [C ] // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2015 : 2625 - 2634 .

AMERINI I , GALTERI L , CALDELLI R , et al . Deepfake video detection through optical flow based CNN [C ] // International Conference on Computer Vision Workshops (ICCVW) . 2019 : 1205 - 1207 .

AFCHAR D , NOZICK V , YAMAGISHI J , et al . Mesonet:a compact facial video forgery detection network [C ] // 2018 IEEE International Workshop on Information Forensics and Security (WIFS) . 2018 : 1 - 7 .

FERNANDES S , RAJ S , ORTIZ E , et al . Predicting heart rate variations of Deepfake videos using neural ODE [C ] // International Conference on Computer Vision Workshops(ICCVW) . 2019 : 1721 - 1729 .

SABIR E , CHENG J , JAISWAL A , et al . Recurrent convolutional strategies for face manipulation detection in videos [C ] // 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2019 : 80 - 87 .

NGUYEN H , FUMING F , YAMAGISHI J , et al . Multitask learning for detecting and segmenting manipulated facial images and videos [J ] . arXiv:abs/1906.06876[astro-ph.CO] .

CHOLLET F , . Xception:deep learning with depth wise separable convolutions [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2017 : 1800 - 1807 .

LI H , CHEN G , LI G , et al . Motion guided attention for video salient object detection [C ] // International Conference on Computer Vision (ICCV) . 2019 : 7273 - 7282 .

LIU H , JIANG B , XIAO Y , et al . Coherent semantic attention for image inpainting [C ] // 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2019 : 4169 - 4178 .

FU J , LIU J , TIAN H , et al . Dual attention network for scene segmentation [C ] // 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2019 : 3146 - 3154 .

HU J , SHEN LI , SUN G . Squeeze-and-excitation networks [C ] // 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2018 : 7132 - 7141 .

WOO S , PARK J , LEE J , et al . CBAM:convolutional block attention module [C ] // The European Conference on Computer Vision(ECCV) . 2018 : 3 - 19 .

Views

609

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰