空域频域相结合的唇型篡改检测方法

林佳滢; 周文柏; 张卫明; 俞能海

doi:10.11959/j.issn.2096-109x.2022075

您当前的位置：

首页 >

文章列表页 >

空域频域相结合的唇型篡改检测方法

学术论文 | 更新时间：2024-06-03

- 空域频域相结合的唇型篡改检测方法
- Lip forgery detection via spatial-frequency domain combination
- 网络与信息安全学报 2022年8卷第6期页码：146-155
- 作者机构：
  
  1. 中国科学院电磁空间信息重点实验室，安徽合肥 230027
  2. 中国科学技术大学网络空间安全学院，安徽合肥 230027
- 作者简介：
  
  [ "林佳滢（1997- ），女，江西赣州人，中国科学技术大学硕士生，主要研究方向为人工智能安全、信息隐藏" ]
  [ "周文柏（1992- ），男，安徽合肥人，中国科学技术大学特任副研究员，主要研究方向为信息隐藏、人工智能安全" ]
  [ "张卫明（1976- ），男，河北定州人，中国科学技术大学教授、博士生导师，主要研究方向为信息隐藏、多媒体内容安全、人工智能安全" ]
  [ "俞能海（1964- ），男，安徽无为人，中国科学技术大学教授、博士生导师，主要研究方向为多媒体信息检索、图像处理与视频通信、数字媒体内容安全" ]
- 基金信息：
  
  国家自然科学基金;The National Natural Science Foundation of China(U20B2047);国家自然科学基金;The National Natural Science Foundation of China(62072421);国家自然科学基金;The National Natural Science Foundation of China(62002334);国家自然科学基金;The National Natural Science Foundation of China(62102386);国家自然科学基金;The National Natural Science Foundation of China(62121002);中国科技大学探索基金项目;Exploration Fund Project of University of Science and Technology of China(YD3480002001);中央高校基础研究基金;Fundamental Research Funds for the Central Universities(WK2100000011)
- DOI：10.11959/j.issn.2096-109x.2022075
  中图分类号： TP309.2
- 网络首发：2022-12，
  
  纸质出版：2022-12-15
- 稿件说明：
移动端阅览
林佳滢, 周文柏, 张卫明, 等. 空域频域相结合的唇型篡改检测方法[J]. 网络与信息安全学报, 2022,8(6):146-155.

Jiaying LIN, Wenbo ZHOU, Weiming ZHANG, et al. Lip forgery detection via spatial-frequency domain combination[J]. Chinese Journal of Network and Information Security, 2022, 8(6): 146-155.
林佳滢, 周文柏, 张卫明, 等. 空域频域相结合的唇型篡改检测方法[J]. 网络与信息安全学报, 2022,8(6):146-155. DOI： 10.11959/j.issn.2096-109x.2022075.

Jiaying LIN, Wenbo ZHOU, Weiming ZHANG, et al. Lip forgery detection via spatial-frequency domain combination[J]. Chinese Journal of Network and Information Security, 2022, 8(6): 146-155. DOI： 10.11959/j.issn.2096-109x.2022075.

摘要

近年来，社交网络中的“换脸”视频层出不穷，对说话者进行唇型篡改是其中的视频代表之一，这给大众生活增添娱乐的同时，对于网络空间中的个人隐私、财产安全也带来了不小隐患。大多数唇型篡改检测方法在无损条件下取得了较好的表现，但广泛存在于社交媒体平台、人脸识别等场景中的压缩操作，在节约像素和时间冗余的同时，会对视频质量造成影响，破坏空域上像素与像素、帧与帧之间的连贯完整性，导致其检测性能的下降，从而引发对真实视频的错判情况。当空域信息无法提供足够有效的特征时，能够抵抗压缩干扰的频域信息就自然而然地成为重点研究对象。针对这一问题，通过分析频率信息在图像结构和梯度反馈上的优势，提出了空域频域相结合的唇型篡改检测方法，有效利用空域、频域信息的各自特点。对于空域上的唇型特征，设计了自适应提取网络和轻量级的注意力模块；对于频域上的频率特征，设计了不同分量的分离提取与融合模块。随后，通过对空域上的唇型特征和频域上的频率特征进行有侧重的融合，保留更多关键纹理信息。此外，在训练中设计细粒度约束，分开真假唇型特征类间距离的同时，拉近类内距离。实验结果表明，得益于频率信息，所提方法能有效改善压缩情况下的检测准确性，并具备一定的迁移性。另外，在对核心模块开展的消融实验中，相关结果验证了频率分量对于抗压缩的有效性，以及双重损失函数在训练中的约束作用。

Abstract

In recent years

numerous “face-swapping” videos have emerged in social networks

one of the representatives is the lip forgery with speakers.While making life more entertaining for the public

it poses a significant crisis for personal privacy and property security in cyberspace.Currently

under non-destructive conditions

most of the lip forgery detection methods achieve good performance.However

the compression operations are widely used in practice especially in social media platforms

face recognition and other scenarios.While saving pixel and time redundancy

the compression operations affect the video quality and destroy the coherent integrity of pixel-to-pixel and frame-to-frame in the spatial domain

and then the degradation of its detection performance and even misjudgment of the real video will be caused.When the information in the spatial domain cannot provide sufficiently effective features

the information in the frequency domain naturally becomes a priority research object because it can resist compression interference.Aiming at this problem

the advantages of frequency information in image structure and gradient feedback were analyzed.Then the lip forgery detection via spatial-frequency domain combination was proposed

which effectively utilized the corresponding characteristics of information in spatial and frequency domains.For lip features in the spatial domain

an adaptive extraction network and a light-weight attention module were designed.For frequency features in the frequency domain

separate extraction and fusion modules for different components were designed.Subsequently

by conducting a weighted fusion of lip features in spatial domain and frequency features in frequency domain

more texture information was preserved.In addition

fine-grained constraints were designed during the training to separate the inter-class distance of real and fake lip features while closing the intra-class distance.Experimental results show that

benefiting from the frequency information

the proposed method can enhance the detection accuracy under compression situation with certain transferability.On the other hand

in the ablation study conducted on the core modules

the results verify the effectiveness of the frequency component for anti-compression and the constraint of the dual loss function in training.

关键词

Keywords

references

SUWAJANAKORN S , SEITZ S M , KEMELMACHER-SHLIZERMAN I , . Synthesizing Obama:learning lip sync from audio [J ] . ACM Transactions on Graphics (TOG) , 2017 , 36 : 1 - 13 .

SIAROHIN A , LATHUILIÈRE S , TULYAKOV S , et al . First order motion model for image animation [J ] . ArXiv , 2019 ,abs/2003.00196.

YI R , YE Z , ZHANG J , et al . Audio-driven talking face video generation with learning-based personalized head pose [J ] . arXiv:2002.10137v2 , 2020 .

PRAJWAL K R , MUKHOPADHYAY R , NAMBOODIRI V P , et al . A lip sync expert is all you need for speech to lip generation in the wild [C ] // Proceedings of the 28th ACM International Conference on Multimedia . 2020 : 484 - 492 .

HALIASSOS A , VOUGIOUKAS K , PETRIDIS S , et al . Lips don't lie:a generalisable and robust approach to face forgery detection [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2021 : 5037 - 5047 .

FARHA Y A , GALL J . MS-TCN:multi-stage temporal convolutional network for action segmentation [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2019 : 3570 - 3579 .

QIAN Y Y , YIN G J , SHENG L , et al . Thinking in frequency:face forgery detection by mining frequency-aware clues [C ] // Proceedings of Computer Vision – ECCV 2020 . 2020 : 86 - 103 .

LI J M , XIE H T , LI J H , et al . Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2021 : 6454 - 6463 .

CHEN S , YAO T P , CHEN Y , et al . Local relation learning for face forgery detection [J ] . arXiv:2105.02577 , 2021 .

REN S Q , HE K M , GIRSHICK R , et al . Faster R-CNN:towards real-time object detection with region proposal networks [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 , 39 ( 6 ): 1137 - 1149 .

HUANG G , LIU Z , VAN DER MAATEN L , et al . Densely connected convolutional networks [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . 2017 : 2261 - 2269 .

SUN Y , WANG X G , TANG X O . Deep learning face representation from predicting 10,000 classes [C ] // Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition . 2014 : 1891 - 1898 .

WEN Y D , ZHANG K P , LI Z F , et al . A discriminative feature learning approach for deep face recognition [C ] // Proceedings of Computer Vision – ECCV 2016 . 2016 : 499 - 515 .

RÖSSLER A , COZZOLINO D , VERDOLIVA L , et al . FaceForensics++:learning to detect manipulated facial images [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . 2019 : 1 - 11 .

LI Y Z , YANG X , SUN P , et al . Celeb-DF:a large-scale challenging dataset for DeepFake forensics [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 3204 - 3213 .

Faceswap . Faceswap github [EB ] .

THIES J , ZOLLHÖFER M , STAMMINGER M , et al . Face 2 face:real-time face capture and reenactment of RGB videos [J ] . ArXiv , 2019 ,abs/2007.14808.

DeepFakes . Deepfakes github [EB ] .

THIES J , ZOLLHÖFER M , NIEßNER M . Real-time expression transfer for facial reenactment [J ] . ACM Transactions on Graphics , 2015 , 34 ( 6 ): 1 - 14 .

LI L Z , BAO J M , ZHANG T , et al . Face X-ray for more general face forgery detection [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 5000 - 5009 .

KINGMA D P , BA J . Adam:a method for stochastic optimization [J ] . CoRR . 2015 ,abs/1412.6980.

FRIDRICH J J , KODOVSKÝ J , . Rich models for steganalysis of digital images [J ] . IEEE Transactions on Information Forensics and Security , 2012 , 7 ( 3 ): 868 - 882 .

AFCHAR D , NOZICK V , YAMAGISHI J , et al . MesoNet:a compact facial video forgery detection network [J ] . 2018 IEEE International Workshop on Information Forensics and Security (WIFS) , 2018 : 1 - 7 .

CHOLLET F , . Xception:deep learning with depthwise separable convolutions [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . 2017 : 1800 - 1807 .

COZZOLINO D , POGGI G , VERDOLIVA L . Recasting residual-based local descriptors as convolutional neural networks:an application to image forgery detection [C ] // Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security . 2017 : 159 - 164 .

BAYAR B , STAMM M C . A deep learning approach to universal image manipulation detection using a new convolutional layer [C ] // Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security . 2016 : 5 - 10 .

RAHMOUNI N , NOZICK V , YAMAGISHI J , et al . Distinguishing computer graphics from natural images using convolution neural networks [J ] . 2017 IEEE Workshop on Information Forensics and Security (WIFS) , 2017 : 1 - 6 .

ZHOU P , HAN X T , MORARIU V I , et al . Two-stream neural networks for tampered face detection [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops . 2017 : 1831 - 1839 .

NGUYEN H H , FANG F M , YAMAGISHI J , et al . Multi-task learning for detecting and segmenting manipulated facial images and videos [C ] // Proceedings of 2019 IEEE 10th International Conference on Biometrics Theory,Applications and Systems . 2019 : 1 - 8 .

MATERN F , RIESS C , STAMMINGER M . Exploiting visual artifacts to expose deepfakesand face manipulations [J ] . 2019 IEEE Winter Applications of Computer Vision Workshops(WACVW) , 2019 : 83 - 92 .

LI Y , LYU S . Exposing DeepFake videos by detecting face warping artifacts [J ] . ArXiv , 2019 ,abs/1811.00656.

浏览量

286

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于双路特征提取和自适应融合的钓鱼网站检测

DNS异常检测：大规模实验分析

FL-IoV：基于联邦学习的综合车联网入侵检测方法

基于分层结构的多任务对抗样本归因方法

彩色图像隐写分析研究进展