Forgery face detection method based on multi-domain temporal features mining

Chuntao ZHU; Chengxi YIN; Bolin ZHANG; Qilin YIN; Wei LU

doi:10.11959/j.issn.2096-109x.2023044

您当前的位置：

首页 >

文章列表页 >

Forgery face detection method based on multi-domain temporal features mining

Papers | 更新时间：2024-06-03

- Forgery face detection method based on multi-domain temporal features mining
- Chinese Journal of Network and Information Security Vol. 9, Issue 3, Pages: 123-134(2023)
- 作者机构：
  
  1. 中山大学计算机学院，广东广州 510006
  2. 广东省信息安全技术重点实验室（中山大学），广东广州 510006
  3. 机器智能与先进计算教育部重点实验室（中山大学），广东广州 510006
  4. 山东省计算机网络重点实验室，山东济南 250014
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2023044
  CLC： TP393
- Online First：2023-06，
  
  Published：25 June 2023
- 稿件说明：
移动端阅览
Chuntao ZHU, Chengxi YIN, Bolin ZHANG, et al. Forgery face detection method based on multi-domain temporal features mining[J]. Chinese Journal of Network and Information Security, 2023, 9(3): 123-134.
DOI：

Chuntao ZHU, Chengxi YIN, Bolin ZHANG, et al. Forgery face detection method based on multi-domain temporal features mining[J]. Chinese Journal of Network and Information Security, 2023, 9(3): 123-134. DOI： 10.11959/j.issn.2096-109x.2023044.

摘要

随着计算机技术在金融服务行业中的不断发展，金融科技便利了人们的日常生活，与此同时，数字金融存在着危害性极大的安全问题。人脸生物信息作为人物身份信息的重要组成部分，广泛应用于金融行业中的支付系统、账号注册等方面；伪造人脸技术的出现不断冲击着数字金融安全体系，给国家资产安全和社会稳定造成了一定的威胁。为了应对伪造人脸带来的安全问题，提出了一种基于多域时序特征挖掘的伪造人脸检测方法。所提方法从视频在空域和频域中存在的时序特征出发，基于人脸统计特征数据分布的一致性以及时间上动作趋势的一致性，对篡改特征进行区分增强。在空域中，所提方法使用改进的长短记忆网络（LSTM）来挖掘帧间的时序特征；在频域中，利用 3D 卷积层来挖掘不同频段频谱的时序信息，并与主干网络提取到的篡改特征进行融合，进而有效地区分伪造人脸和真实人脸。所提方法在主流数据集中表现优越，证明了所提方法的有效性。

Abstract

Financial technology has greatly facilitated people’s daily life with the continuous development of computer technology in the financial services industry.However

digital finance is accompanied by security problems that can be extremely harmful.Face biometrics

as an important part of identity information

is widely used in payment systems

account registration

and many other aspects of the financial industry.The emergence of face forgery technology constantly impacts the digital financial security system

posing a threat to national asset security and social stability.To address the security problems caused by fake faces

a forgery face detection method based on multi-domain temporal features mining was proposed.The tampering features were distinguished and enhanced based on the consistency of statistical feature data distribution and temporal action trend in the temporal features of videos existing in the spatial domain and frequency domain.Temporal information was mined in the spatial domain using an improved LSTM

while in the frequency domain

temporal information existing in different frequency bands of the spectrum was mined using 3D convolution layers.The information was then fused with the tampering features extracted from the backbone network

thus effectively distinguishing forged faces from real ones.The effectiveness of the proposed method was demonstrated on mainstream datasets.

关键词

Keywords

references

DeepFakes GitHub non official project based on original deepfakes thread [EB ] .

THIES J , ZOLLHÖFER M , STAMMINGER M , et al . Face2Face:real-time face capture and reenactment of RGB videos [C ] // Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2016 : 2387 - 2395 .

FaceSwap GitHub non official project based on original deepfakes thread [EB ] .

GOODFELLOW I J , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial networks [J ] . arXiv Preprint arXiv:1406.2661 . 2014 .

XU W J , KESHMIRI S , WANG G H . Adversarially approximated autoencoder for image generation and manipulation [J ] . IEEE Transactions on Multimedia , 2019 , 21 ( 9 ): 2387 - 2396 .

王鹏程 , 郑海斌 , 邹健飞 , 等 . 面向商用活体检测平台的鲁棒性评估 [J ] . 网络与信息安全学报 , 2022 , 8 ( 1 ): 180 - 189 .

WANG P C , ZHENG H B , ZOU J F , et al . Robustness evaluation of commercial liveness detection platform [J ] . Chinese Journal of Network and Information Security , 2022 , 8 ( 1 ): 180 - 189 .

MATERN F , RIESS C , STAMMINGER M . Exploiting visual artifacts to expose deepfakes and face manipulations [C ] // Proceedings of 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) . 2019 : 83 - 92 .

PENG B , WANG W , DONG J , et al . Optimized 3D lighting environment estimation for image forgery detection [J ] . IEEE Transactions on Information Forensics and Security , 2017 , 12 ( 2 ): 479 - 494 .

NIRKIN Y , WOLF L , KELLER Y , et al . DeepFake detection based on discrepancies between faces and their context [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 10 ): 6111 - 6121 .

LIU H G , LI X D , ZHOU W B , et al . Spatial-phase shallow learning:rethinking face forgery detection in frequency domain [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2021 : 772 - 781 .

林佳滢 , 周文柏 , 张卫明 , 等 . 空域频域相结合的唇型篡改检测方法 [J ] . 网络与信息安全学报 , 2022 , 8 ( 6 ): 146 - 155 .

LIN J Y , ZHOU W B , ZHANG W M , et al . Lip forgery detection via spatial-frequency domain combination [J ] . Chinese Journal of Network and Information Security , 2022 , 8 ( 6 ): 146 - 155 .

LI Y Z , CHANG M C , LYU S W . Exposing AI created fake videos by detecting eye blinking [C ] // Proceedings of 2018 IEEE International Workshop on Information Forensics and Security (WIFS) . 2019 : 1 - 7 .

SUN Z K , HAN Y J , HUA Z Y , et al . Improving the efficiency and robustness of deepfakes detection through precise geometric features [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2021 : 3608 - 3617 .

HOCHREITER S , SCHMIDHUBER J . Long short-term memory [J ] . Neural Computation , 1997 , 9 ( 8 ): 1735 - 1780 .

WANG Y B , WU H X , ZHANG J J , et al . PredRNN:a recurrent neural network for spatiotemporal predictive learning [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023 , 45 ( 2 ): 2208 - 2225 .

THIES J , ZOLLHÖFER M , NIEßNER M . Deferred neural rendering [J ] . ACM Transactions on Graphics , 2019 , 38 ( 4 ): 1 - 12 .

LIU M , BREUEL T , KAUTZ J . Unsupervised image-to-image translation networks [J ] . arXiv Preprint arXiv:1703.00848 , 2017 .

KARRAS T , AILA T , LAINE S , et al . Progressive growing of gans for improved quality,stability,and variation [J ] . arXiv Preprint arXiv:1710.10196 , 2017 .

KARRAS T , LAINE S , AILA T M . A style-based generator architecture for generative adversarial networks [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 4396 - 4405 .

HUANG X , BELONGIE S . Arbitrary style transfer in real-time with adaptive instance normalization [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . 2017 : 1510 - 1519 .

LI L Z , BAO J M , YANG H , et al . Advancing high fidelity identity swapping for forgery detection [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 5073 - 5082 .

PAN X Y , ZHANG X , LYU S W . Exposing image splicing with inconsistent local noise variances [C ] // Proceedings of 2012 IEEE International Conference on Computational Photography (ICCP) . 2012 : 1 - 10 .

FERRARA P , BIANCHI T , DE ROSA A , et al . Image forgery localization via fine-grained analysis of CFA artifacts [J ] . IEEE Transactions on Information Forensics and Security , 2012 , 7 ( 5 ): 1566 - 1577 .

FRIDRICH J , KODOVSKY J . Rich models for steganalysis of digital images [J ] . IEEE Transactions on Information Forensics and Security , 2012 , 7 ( 3 ): 868 - 882 .

YANG X , LI Y Z , LYU S W . Exposing deep fakes using inconsistent head poses [C ] // Proceedings of ICASSP 2019 - 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP) . 2019 : 8261 - 8265 .

AGARWAL S , FARID H , GU Y , et al . Protecting world leaders against deep fakes [C ] // Proceedings of CVPR Workshops . 2019 ,1:38.

CHOLLET F . Xception:deep learning with depthwise separable convolutions [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2017 : 1800 - 1807 .

AFCHAR D , NOZICK V , YAMAGISHI J , et al . MesoNet:a compact facial video forgery detection network [C ] // Proceedings of 2018 IEEE International Workshop on Information Forensics and Security (WIFS) . 2019 : 1 - 7 .

NGUYEN H H , YAMAGISHI J , ECHIZEN I . Capsule-forensics:using capsule networks to detect forged images and videos [C ] // Proceedings of ICASSP 2019 - 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP) . 2019 : 2307 - 2311 .

LI L Z , BAO J M , ZHANG T , et al . Face X-ray for more general face forgery detection [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 5000 - 5009 .

DURALL R , KEUPER M , PFREUNDT F J , et al . Unmasking DeepFakes with simple features [J ] . arXiv Preprint arXiv:1911.00686 , 2019 .

QIAN Y Y , YIN G J , SHENG L , et al . Thinking in frequency:face forgery detection by mining frequency-aware clues [C ] // Proceedings of European Conference on Computer Vision . 2020 : 86 - 103 .

GÜERA D , DELP E J . Deepfake video detection using recurrent neural networks [C ] // Proceedings of 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) . 2019 : 1 - 6 .

HU J , LIAO X , WANG W , et al . Detecting compressed deepfake videos in social networks using frame-temporality two-stream convolutional network [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2022 , 32 ( 3 ): 1089 - 1102 .

LU W , LIU L Y , ZHANG B L , et al . Detection of deepfake videos using long-distance attention [J ] . IEEE Transactions on Neural networks and Learning Systems , 2023 ,( 99 ): 1 - 14 .

SHI X J , CHEN Z R , WANG H , et al . Convolutional LSTM Network:a machine learning approach for precipitation nowcasting [C ] // Proceedings of the 28th International Conference on Neural Information Processing System . 2015 : 802 - 810 .

JADERBERG M , SIMONYAN K , ZISSERMAN A , et al . Spatial transformer networks [J ] . arXiv Preprint arXiv:1506.02025 , 2015 .

WANG Q L , WU B G , ZHU P F , et al . ECA-net:efficient channel attention for deep convolutional neural networks [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 11531 - 11539 .

RÖSSLER A , COZZOLINO D , VERDOLIVA L , et al . FaceForensicss:learning to detect manipulated facial images [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . 2020 : 1 - 11 .

LI Y Z , YANG X , SUN P , et al . Celeb-DF:a large-scale challenging dataset for DeepFake forensics [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 3204 - 3213 .

DOLHANSKY B , HOWES R , PFLAUM B , et al . The deepfake detection challenge (DFDC) preview dataset [J ] . arXiv Preprint arXiv:1910.08854 , 2019 .

WANG C R , DENG W H . Representative forgery mining for fake face detection [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2021 : 14918 - 14927 .

ZHAO H Q , WEI T Y , ZHOU W B , et al . Multi-attentional deepfake detection [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2021 : 2185 - 2194 .

Views

230

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Noise-attention-based forgery face detection method

Related Author

Bolin ZHANG

Chuntao ZHU

Qilin YIN

Jingqiao FU

Lingyi LIU

Jiarui LIU

Hongmei LIU

Wei LU

Related Institution

Guangdong Province Key Laboratory of Information Security Technology

Ministry of Education Key Laboratory of Machine Intelligence and Advanced Computing

⁰