基于双层注意力的Deepfake换脸检测

龚晓娟; 黄添强; 翁彬; 叶锋; 徐超; 游立军

doi:10.11959/j.issn.2096-109x.2021032

您当前的位置：

首页 >

文章列表页 >

基于双层注意力的Deepfake换脸检测

学术论文 | 更新时间：2024-06-03

- 基于双层注意力的Deepfake换脸检测
- Deepfake swapped face detection based on double attention
- 网络与信息安全学报 2021年7卷第2期页码：151-160
- 作者机构：
  
  1. 福建师范大学数学与信息学院，福建福州 350117
  2. 数字福建大数据安全技术研究所，福建福州 350117
  3. 福建省灾害天气重点实验室，福建福州 350001
- 作者简介：
  
  [ "龚晓娟（1995- ），女，福建福州人，福建师范大学硕士生，主要研究方向为数字多媒体取证。" ]
  [ "黄添强（1971- ），男，福建仙游人，博士，福建师范大学教授、博士生导师，主要研究方向为机器学习、数字多媒体取证。" ]
  [ "翁彬（1981- ），男，福建福州人，博士，福建师范大学讲师，主要研究方向为机器学习及应用。" ]
  [ "叶锋（1978- ），男，福建福州人，博士，福建师范大学副教授，主要研究方向为多媒体信号处理、计算机视觉。" ]
  [ "徐超（1981- ），男，湖北天门人，福建师范大学讲师，主要研究方向为视频篡改检测。" ]
  [ "游立军（1974- ），男，福建莆田人，福建省灾害天气重点实验室高级工程师，主要研究方向为气候数据分析。" ]
- 基金信息：
  
  国家重点研发计划专项基金资助项目;National Key Program for Developing Basic Science(2018YFC1505805);国家自然科学基金;The National Natural Science Foundation of China(62072106);国家自然科学基金资助项目;The National Natural Science Foundation of China(61070062);应用数学福建省高校重点实验室基金资助项目;Applied Mathematics Fujian Provincial Key Laboratory Project(SX201803)
- DOI：10.11959/j.issn.2096-109x.2021032
  中图分类号： TP393
- 网络首发：2021-04，
  
  纸质出版：2021-04-15
- 稿件说明：
移动端阅览
龚晓娟, 黄添强, 翁彬, 等. 基于双层注意力的Deepfake换脸检测[J]. 网络与信息安全学报, 2021,7(2):151-160.

Xiaojuan GONG, Tianqiang HUANG, Bin WENG, et al. Deepfake swapped face detection based on double attention[J]. Chinese Journal of Network and Information Security, 2021, 7(2): 151-160.
龚晓娟, 黄添强, 翁彬, 等. 基于双层注意力的Deepfake换脸检测[J]. 网络与信息安全学报, 2021,7(2):151-160. DOI： 10.11959/j.issn.2096-109x.2021032.

Xiaojuan GONG, Tianqiang HUANG, Bin WENG, et al. Deepfake swapped face detection based on double attention[J]. Chinese Journal of Network and Information Security, 2021, 7(2): 151-160. DOI： 10.11959/j.issn.2096-109x.2021032.

摘要

针对现有Deepfake检测算法中普遍存在的准确率低、可解释性差等问题，提出融合双层注意力的神经网络模型，该模型利用通道注意力捕获假脸的异常特征，并结合空间注意力聚焦异常特征的位置，充分学习假脸异常部分的上下文语义信息，从而提升换脸检测的有效性和准确性。并以热力图的形式有效地展示了真假脸的决策区域，使换脸检测模型具备一定程度的解释性。在 FaceForensics++开源数据集上的实验表明，所提方法的检测精度优于MesoInception、Capsule-Forensics和XceptionNet检测方法。

Abstract

In view of the existing Deepfake detection algorithms

such problems as low accuracy and poor interpretability are common.A neural network model combining the double attention was proposed

which used channel attention to capture the abnormal features of false faces and combined the location of spatial attention to focus the abnormal features.To fully learn the contextual semantic information of the abnormal part of the false face

so as to improve the effectiveness and accuracy of face changing detection.In addition

the decision-making area of real and fake faces was shown effectively in the form of thermal diagram

which provided a certain degree of explanation for the face exchange detection model.Experiments on FaceForensics ++ open source data set show that the detection accuracy of proposed method is superior to MesoInception

Capsule-Forensics and XceptionNet.

关键词

Keywords

references

THIES J , ZOLLHOFER M , STAMMINGER M , et al . Demo of Face2Face:real-time face capture and reenactment of RGB videos [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2016 ( 1 ): 2387 - 2395 .

RÖSSLER A , COZZOLINOD , VERDOLIVAL , , et al . FaceForensics++:learning to detect manipulated facial images [C ] // International Conference on Computer Vision(ICCV) . 2019 : 1 - 11 .

LI Y , CHANG M , FARID H , et al . In ictu.oculi:exposing AI generated fake face videos by detecting eye blinking [C ] // International Workshop on Information Forensics and Security(WIFS) . 2018 : 1 - 7 .

COZZOLINO D , VERDOLIVA L . Noiseprint:a CNN-based camera model fingerprint [J ] . IEEE Transactions on Information Forensics and Security , 2020 ( 15 ): 144 - 159 .

DONG H , LIU F , STEHOUWER J , et al . On the detection of digital face manipulation [J ] . arXiv:1910.01717[astro-ph.CO] .

ZHOU C , WU M , LAM S . SSA-CNN:semantic self-attention CNN for pedestrian detection [J ] . arXiv:1902.09080v1[astro-ph.CO] .

NGUYEN H , YAMAGISHI J , ECHIZEN I . Use of a capsule network to detect fake images and videos [J ] . arXiv:1910.12467v2[astro-ph.CO] .

SELVARAJU M , COGSWELL M , DAS A . Grad-CAM:visual explanations from deep networks via gradient-based localization [C ] // International Conference on Computer Vision (ICCV) . 2017 : 618 - 626 .

YANG X , LI Y , LYU S . Exposing deep fakes using inconsistent head poses [C ] // 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP) . 2019 : 8261 - 8265 .

MATERN F , RIESS C , STAMMINGER M . Exploiting visual artifacts to expose Deepfakes and face manipulations [C ] // 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) . 2019 : 83 - 92 .

NATARAJ L , MOHAMMED T M , MANJUNATH B S , et al . Detecting GAN generated fake images using co-occurrence matrices [C ] // Media Watermarking,Security,and Forensics . 2019 .

LI Y , LYU S . Exposing Deepfake videos by detecting face warping artifacts [C ] // 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2019 : 46 - 52 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [C ] // 3rd International Conference on Learning Representations (ICLR) . 2015 .

HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . 2016 : 770 - 778 .

GUERA D , DELP E . Deepfake video detection using recurrent neural networks [C ] // 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) . 2018 : 1 - 6 .

DONAHUE J , HENDRICKS L , GUADARRAMA S , et al . Long-term recurrent convolutional networks for visual recognition and description [C ] // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2015 : 2625 - 2634 .

AMERINI I , GALTERI L , CALDELLI R , et al . Deepfake video detection through optical flow based CNN [C ] // International Conference on Computer Vision Workshops (ICCVW) . 2019 : 1205 - 1207 .

AFCHAR D , NOZICK V , YAMAGISHI J , et al . Mesonet:a compact facial video forgery detection network [C ] // 2018 IEEE International Workshop on Information Forensics and Security (WIFS) . 2018 : 1 - 7 .

FERNANDES S , RAJ S , ORTIZ E , et al . Predicting heart rate variations of Deepfake videos using neural ODE [C ] // International Conference on Computer Vision Workshops(ICCVW) . 2019 : 1721 - 1729 .

SABIR E , CHENG J , JAISWAL A , et al . Recurrent convolutional strategies for face manipulation detection in videos [C ] // 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2019 : 80 - 87 .

NGUYEN H , FUMING F , YAMAGISHI J , et al . Multitask learning for detecting and segmenting manipulated facial images and videos [J ] . arXiv:abs/1906.06876[astro-ph.CO] .

CHOLLET F , . Xception:deep learning with depth wise separable convolutions [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2017 : 1800 - 1807 .

LI H , CHEN G , LI G , et al . Motion guided attention for video salient object detection [C ] // International Conference on Computer Vision (ICCV) . 2019 : 7273 - 7282 .

LIU H , JIANG B , XIAO Y , et al . Coherent semantic attention for image inpainting [C ] // 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2019 : 4169 - 4178 .

FU J , LIU J , TIAN H , et al . Dual attention network for scene segmentation [C ] // 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2019 : 3146 - 3154 .

HU J , SHEN LI , SUN G . Squeeze-and-excitation networks [C ] // 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2018 : 7132 - 7141 .

WOO S , PARK J , LEE J , et al . CBAM:convolutional block attention module [C ] // The European Conference on Computer Vision(ECCV) . 2018 : 3 - 19 .

浏览量

609

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据