Robust deepfake detection method based on siamese network

LIN Shanhe

doi:10.11959/j.issn.2096-109x.2024016

您当前的位置：

首页 >

文章列表页 >

Robust deepfake detection method based on siamese network

Papers | 更新时间：2024-08-14

- Robust deepfake detection method based on siamese network
- Chinese Journal of Network and Information Security Vol. 10, Issue 2, Pages: 182-189(2024)
- 作者机构：
  
  福建星网锐捷通讯股份有限公司，福建福州 350117
- 作者简介：
- 基金信息：
  
  The Innovation Strategic Research Plan Project of Fujian Province(2023R0156)
- DOI：10.11959/j.issn.2096-109x.2024016
  CLC： TP393
- Received：05 May 2023，
  
  Revised：2024-03-03，
  
  Published：15 April 2024
- 稿件说明：
移动端阅览
林善和. 基于孪生网络的鲁棒性深度伪造检测方法[J]. 网络与信息安全学报, 2024, 10(02): 182-189.

Format: LIN S H. Robust deepfake detection method based on siamese network[J]. Chinese Journal of Network and Information Security, 2024, 10(02): 182-189.
林善和. 基于孪生网络的鲁棒性深度伪造检测方法[J]. 网络与信息安全学报, 2024, 10(02): 182-189. DOI： 10.11959/j.issn.2096-109x.2024016.

Format: LIN S H. Robust deepfake detection method based on siamese network[J]. Chinese Journal of Network and Information Security, 2024, 10(02): 182-189. DOI： 10.11959/j.issn.2096-109x.2024016.

摘要

目前，使用深度伪造技术合成伪造人脸图像的案例非常广泛，深度伪造技术通过对人脸图像进行面部替换或表情更改，从而实现深度伪造的目的。这类深度伪造图像的肆意传播，可能对社会和个人造成许多不良后果。因此，不少学者着手研究深度伪造检测方法。虽然现有的检测方法在高质量图像的检测上能实现非常高的准确率，但是在检测经过图像压缩操作的低质量人脸图像时，其检测精度会大幅下降。研究旨在改进现有深度伪造检测方法鲁棒性不足的问题。为此，提出了一种基于孪生网络的深度伪造检测方法，其思想是使用孪生网络来学习高质量图像和低质量图像之间公共伪造特征，通过牺牲部分高质量图像的特征提取能力，来提高网络对低质量图像的表征能力，从而使网络在不同压缩率伪造图像的检测上都具有较高的准确率。实验结果表明，所提方法在多个不同压缩率数据集上的综合平均准确率达到90%以上，优于多个现有检测方法。通过消融实验证明所提方法简单有效，且适用于不同的主干网络。

Abstract

The proliferation of deepfake (DF) technology for generating manipulated facial expressions in synthetic images has raised concerns due to its potential negative impacts on individuals and society. In response to the need for robust detection

researchers have been developing methods to identify deepfakes. While current detection methods perform well on high-quality images

they often falter when confronted with low-quality or compressed images. This study focused on enhancing the robustness of deepfake detection methods to address these limitations. A novel approach leveraging a Siamese network was proposed

designed to learn common forgery features across both high-quality and low-quality images. This was achieved by trading off some of the high-quality image feature extraction capabilities to bolster the representational capacity for low-quality images. The proposed method demonstrated an average accuracy exceeding 90% across various datasets with different compression levels

surpassing several existing detection techniques. The simplicity

effectiveness

and adaptability of the proposed method to different backbone networks were further substantiated through ablation experiments.

关键词

Keywords

references

CHOPRA S , HADSELL R , LECUN Y . Learning a similarity metric discriminatively, with application to face verification [C ] // Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . 2005 : 539 - 546 .

RÖSSLER A , COZZOLINO D , VERDOLIVA L , et al . FaceForensics: learning to detect manipulated facial images [C ] // Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision . 2019 : 1 - 11 .

CHOLLET F . Xception: deep learning with depthwise separable convolutions [C ] // Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition . 2017 : 1800 - 1807 .

NGUYEN H H , YAMAGISHI J , ECHIZEN I . Use of a capsule network to detect fake images and videos [J ] . arXiv preprint arXiv: 1910. 12467 . 2019 .

WANG J K , WU Z X , OUYANG W H , et al . M2TR: multi-modal multi-scale transformers for Dee fake detection [C ] // Proceedings Proceedings of the 2022 International Conference on Multimedia Retrieval . 2022 : 615 - 623 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [J ] . arXiv preprint arXiv: 1706. 03762 . 2017 .

COCCOMINI D A , MESSINA N , GENNARO C , et al . Combining EfficientNet and vision transformers for video deepfake detection [C ] // Proceedings of the International Conference on Image Analysis and Processing . 2022 : 219 - 229 .

WANG C R , DENG W H . Representative forgery mining for fake face detection [C ] // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2021 : 14918 - 14927 .

CHEN L , ZHANG Y , SONG Y B , et al . Self-supervised learning of adversarial example: towards good generalizations for deepfake detection [C ] // Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2022 : 18689 - 18698 .

WANG B , LI Y C , WU X H , et al . Face forgery detection based on the improved Siamese network [J ] . Security and Communication Networks , 2022 : 5169873 .

LIU H G , LI X D , ZHOU W B , et al . Spatial-phase shallow learning: rethinking face forgery detection in frequency domain [C ] // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2021 : 772 - 781 .

WANG G J , JIANG Q , JIN X , et al . MC-LCR: Multimodal contrastive classification by locally correlated representations for effective face forgery detection [J ] . Knowledge-Based Systems , 2022 , 250 : 109114 .

吴文轩 , 周文柏 , 张卫明 , 等 . 基于块间光照不一致性的深度伪造检测算法 [J ] . 网络与信息安全学报 , 2023 , 9 ( 1 ): 167 - 177 .

WU W X , ZHOU W B , ZHANG W M , et al . Deepfake detection method based on patch-wise lighting inconsistency [J ] . Chinese Journal of Network and Information Security , 2023 , 9 ( 1 ): 167 - 177 .

朱春陶 , 尹承禧 , 张博林 , 等 . 基于多域时序特征挖掘的伪造人脸检测方法 [J ] . 网络与信息安全学报 , 2023 , 9 ( 3 ): 123 - 134 .

ZHU C T , YIN C X , ZHANG B L , et al . Forgery face detection method based on multi-domain temporal features mining [J ] . Chinese Journal of Network and Information Security , 2023 , 9 ( 3 ): 123 - 134 .

THIES J , ZOLLHÖFER M , STAMMINGER M , et al . Face2Face: real-time face capture and reenactment of RGB videos [C ] // Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 2387 - 2395 .

THIES J , ZOLLHÖFER M , NIEßNER M . Deferred neural rendering: image synthesis using neural textures [J ] . ACM Transactions on Graphics , 38 ( 4 ): 66 .

KING D E . Dlib-ml: a machine learning toolkit [J ] . Journal of Machine Learning Research . 2009 , 10 : 1755 - 1758 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C ] // Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 770 - 778 .

Views

138

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Robust Schnorr-based subgroup multi-signature scheme

Deepfake detection method based on patch-wise lighting inconsistency

Robust reinforcement learning algorithm based on pigeon-inspired optimization

Research on the robustness of convolutional neural networks in image recognition

Robustness evaluation of commercial liveness detection platform

Related Author

WANG Zhiwei

ZHU Qiuchi

ZHANG Zhenqi

Nenghai YU

Weiming ZHANG

Wenbo ZHOU

Wenxuan WU

Mohong ZHENG

Related Institution

School of Computer Science, Nanjing University of Posts and Telecommunications

Jiangsu Key Laboratory of Big Data Security and Intelligent Processing, Nanjing University of Posts and Telecommunications

China Electronics Standardization Institute

College of Astronautics, Nanjing University of Aeronautics and Astronautics

The 7th Research Institute of China Electronics Technology Group Corporation

⁰