基于强化样本的伪孪生网络图像篡改定位模型

王金伟; 张子荷; 罗向阳; 马宾

doi:10.11959/j.issn.2096-109x.2024010

您当前的位置：

首页 >

文章列表页 >

基于强化样本的伪孪生网络图像篡改定位模型

学术论文 | 更新时间：2024-06-03

- 基于强化样本的伪孪生网络图像篡改定位模型
- Pseudo-siamese network image tampering localization model based on reinforced samples
- 网络与信息安全学报 2024年10卷第1期页码：33-47
- 作者机构：
  
  1. 南京信息工程大学数字取证教育部工程研究中心，江苏南京210044
  2. 南京信息工程大学计算机学院，江苏南京210044
  3. 南京信息工程大学网络空间安全学院，江苏南京210044
  4. 数字工程与先进计算国家重点实验室，河南郑州450001
  5. 信息工程大学，河南郑州 450001
  6. 齐鲁工业大学网络空间安全学院，山东济南 250353
- 作者简介：
  
  [ "王金伟（1978− ），男，内蒙古呼伦贝尔人，南京信息工程大学教授、博士生导师，主要研究方向为人工智能安全、多媒体取证和信息隐藏" ]
  [ "张子荷（1999− ），女，河北廊坊人，南京信息工程大学硕士生，主要研究方向为信息安全、多媒体取证" ]
  [ "罗向阳（1978− ），男，湖北荆门人，信息工程大学教授、博士生导师，主要研究方向为图像隐写、隐写分析技术" ]
  [ "马宾（1973− ），男，山东济宁人，齐鲁工业大学教授、博士生导师，主要研究方向为可逆信息隐藏、多媒体取证、隐写与隐写分析" ]
- 基金信息：
  
  国家自然科学基金;The National Natural Science Foundation of China(62072250);国家自然科学基金;TheNational Natural Science Foundation of China(62172435);国家自然科学基金;The National Natural Science Foundation of China(U1804263);国家自然科学基金;The National Natural Science Foundation of China(U20B2065)
- DOI：10.11959/j.issn.2096-109x.2024010
  中图分类号： TP391
- 网络首发：2024-02，
  
  纸质出版：2024-02-25
- 稿件说明：
移动端阅览
王金伟, 张子荷, 罗向阳, 等. 基于强化样本的伪孪生网络图像篡改定位模型[J]. 网络与信息安全学报, 2024,10(1):33-47.

Jinwei WANG, Zihe ZHANG, Xiangyang LUO, et al. Pseudo-siamese network image tampering localization model based on reinforced samples[J]. Chinese Journal of Network and Information Security, 2024, 10(1): 33-47.
王金伟, 张子荷, 罗向阳, 等. 基于强化样本的伪孪生网络图像篡改定位模型[J]. 网络与信息安全学报, 2024,10(1):33-47. DOI： 10.11959/j.issn.2096-109x.2024010.

Jinwei WANG, Zihe ZHANG, Xiangyang LUO, et al. Pseudo-siamese network image tampering localization model based on reinforced samples[J]. Chinese Journal of Network and Information Security, 2024, 10(1): 33-47. DOI： 10.11959/j.issn.2096-109x.2024010.

摘要

随着互联网不断发展，网络上的篡改图像越来越多，掩盖篡改痕迹的手段越来越丰富。而现在大多数检测模型没有考虑到图像后处理操作对篡改检测算法的影响，限制了其在实际生活中的应用。为了解决上述问题，提出了一种通用的基于强化样本的伪孪生网络图像篡改定位模型。所提模型利用伪孪生网络，一方面学习真实图像中的篡改特征；另一方面通过约束卷积，抑制图像内容，从而能够更加关注篡改残留的痕迹信息。网络的两分支结构可以达到充分利用图像特征信息的目的。模型利用强化样本，可以自适应地生成当前最需要学习的篡改类型图片，实现对模型有针对性地训练，使得模型在各个方向上学习收敛，最终得到全局最优模型。利用数据增强思路，自动生成丰富的篡改图像以及其对应的掩膜，这很好地解决了篡改数据集有限的问题。在 4 个数据集上的大量实验证明了所提模型在像素级操作检测方面的可行性和有效性。尤其是在Columbia数据集上，算法的F1值提高了33.5%，Matthews correlation coefficirnt（MCC）得分提高了 23.3%，说明所提模型利用深度学习模型的优点，显著提高了篡改定位的检测效果。

Abstract

With the continuous development of the internet

an increasing number of images have been tampered with on the network

accompanied by a growing range of techniques to cover up tampering traces.However

most current detection models neglect the impact of image post-processing on tamper detection algorithms

limiting their real-life applications.To address these issues

a general image tampering location model based on enhanced samples and the pseudo-twin network was proposed.The pseudo-twin network enabled the model to learn tampering features in real images.On one hand

by applying convolution constraints

the image content was suppressed

allowing the model to focus more on residual trace information of tampering.The two-branch structure of the network facilitated the comprehensive utilization of image feature information.By utilizing enhanced samples

the model could dynamically generate the most crucial pictures for learning tamper types

enabling targeted training of the model.This approach ensured that the model converged in all directions

ultimately obtaining the global optimal model.The idea of data enhancement was employed to automatically generate abundant tampered images and corresponding masks

effectively resolving the limited tampering dataset issue.Extensive experiments were conducted on four datasets

demonstrating the feasibility and effectiveness of the proposed model in pixel-level tamper detection.Particularly on the Columbia dataset

the algorithm achieves a 33.5% increase in F1 score and a 23.3% increase in MCC score.These results indicate that the proposed model harnesses the advantages of deep learning models and significantly improves the effectiveness of tamper location detection.

关键词

Keywords

references

FERRARA P , BIANCHI T , DE ROSA A , et al . Image forgery localization via fine-grained analysis of CFA artifacts [J ] . IEEE Transactions on Information Forensics and Security , 2012 , 7 ( 5 ): 1566 - 1577 .

LI C , MA Q , XIAO L M , et al . An image copy move forgery detection method using QDCT [C ] // Proceedings of the International Conference on Internet Multimedia Computing and Service . 2016 : 5 - 8 .

AMBILI B , GEORGE N . A robust technique for splicing detection in tampered blurred images [C ] // Proceedings of 2017 International Conference on Trends in Electronics and Informatics (ICEI) . 2017 : 897 - 901 .

黄维 , 黄添强 , 张雪莉 , 等 . 基于块效应网格偏移的重获取JPEG图像篡改检测 [J ] . 网络与信息安全学报 , 2017 , 3 ( 12 ): 24 - 30 .

HUANG W , HUANG T Q , ZHANG X L , et al . JPEG recapture image tamper detection method based on block effect grid offset [J ] . Chinese Journal of Network and Information Security , 2017 , 3 ( 12 ): 24 - 30 .

BAYAR B , STAMM M C . Constrained convolutional neural networks:a new approach towards general purpose image manipulation detection [J ] . IEEE Transactions on Information Forensics and Security , 2018 , 13 ( 11 ): 2691 - 2706 .

WU Y , ABD-ALMAGEED W , NATARAJAN P . Image copy-move forgery detection via an end-to-end deep neural network [C ] // Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) . 2018 : 1907 - 1915 .

蒋小玉 , 刘春晓 . 边缘与区域不一致性引导下的图像拼接篡改检测网络 [J ] . 中国图象图形学报 , 2021 , 26 ( 10 ): 2411 - 2420 .

JIANG X Y , LIU C X . Edge and region inconsistency-guided image splicing tamper detection network [J ] . Journal of Image and Graphics , 2021 , 26 ( 10 ): 2411 - 2420 .

朱新同 , 唐云祁 , 耿鹏志 . 基于特征融合的篡改与深度伪造图像检测算法 [J ] . 信息网络安全 , 2021 ( 8 ): 70 - 81 .

ZHU X T , TANG Y Q , GENG P Z . Detection algorithm of tamper and deepfake image based on feature fusion [J ] . Netinfo Security , 2021 ( 8 ): 70 - 81 .

李小红 , 王晓霞 . 一种剪枝压缩CNN移动设备端图像篡改检测方法 [J ] . 计算机仿真 , 2021 , 38 ( 3 ): 83 - 86 , 91 .

LI X H , WANG X X . An image tamper detection method for CNN mobile devices based on pruning compression [J ] . Computer Simulation , 2021 , 38 ( 3 ): 83 - 86 , 91 .

何家豪 . 图像复制移动篡改检测研究及实现 [D ] . 大连:大连理工大学 , 2021 .

HE J H . Research and implementation of image copy mobile tampering detection [D ] . Dalian:Dalian University of Technology , 2021 .

ZHUANG P Y , LI H D , TAN S Q , et al . Image tampering localization using a dense fully convolutional network [J ] . IEEE Transactions on Information Forensics and Security , 2021 , 16 : 2986 - 2999 .

WEI Y , MA J F , WANG Z Z , et al . Image splicing forgery detection by combining synthetic adversarial networks and hybrid dense U-net based on multiple spaces [J ] . International Journal of Intelligent Systems , 2022 , 37 ( 11 ): 8291 - 8308 .

LU W , XU W B , SHENG Z Q . An interpretable image tampering detection approach based on cooperative game [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2023 , 33 ( 2 ): 952 - 962 .

ZHAO T C , XU X , XU M Z , et al . Learning self-consistency for deepfake detection [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . 2021 : 15003 - 15013 .

CHEN L , ZHANG Y , SONG Y B , et al . Self-supervised learning of adversarial example:towards good generalizations for deepfake detection [C ] // Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2022 : 18689 - 18698 .

SOLIMAN M , LEHMAN C , ALREGIB G . S6:semi-supervised self-supervised semantic segmentation [C ] // Proceedings of 2020 IEEE International Conference on Image Processing (ICIP) . 2020 : 1861 - 1865 .

XU H , LI X , GONG W Y . Research on recognition of landslides with remote sensing images based on extreme learning machine [C ] // Proceedings of 2017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC) . 2017 : 740 - 747 .

HALIASSOS A , MIRA R , PETRIDIS S , et al . Leveraging real talking faces via self-supervision for robust forgery detection [C ] // Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2022 : 14930 - 14942 .

KRISHNA V A , REDDY A , NAGAJYOTHI D . Signature recognition using Siamese neural networks [C ] // Proceedings of 2021 IEEE International Conference on Mobile Networks and Wireless Communications (ICMNWC) . 2022 : 1 - 4 .

WANG Y Y , ZHANG W S , ZHANG L M , et al . Depthwise over-parameterized Siamese network for visual tracking [C ] // Proceedings of 2021 International Conference on Information Technology and Biomedical Engineering (ICITBE) . 2022 : 58 - 62 .

SUN K , YAO T P , CHEN S , et al . Dual contrastive learning for general face forgery detection [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 2 ): 2316 - 2324 .

COZZOLINO D , POGGI G , VERDOLIVA L . Splicebuster:a new blind image splicing detector [C ] // Proceedings of 2015 IEEE International Workshop on Information Forensics and Security (WIFS) . 2015 : 1 - 6 .

ZHOU P , HAN X T , MORARIU V I , et al . Learning rich features for image manipulation detection [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2018 : 1053 - 1061 .

ZHAO Q , CAO G , ZHOU A T , et al . Image tampering detection via semantic segmentation network [C ] // Proceedings of 2020 15th IEEE International Conference on Signal Processing (ICSP) . 2020 : 165 - 169 .

CHEN J X , LIAO X , WANG W , et al . SNIS:a signal noise separation-based network for post-processed image forgery detection [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2023 , 33 ( 2 ): 935 - 951 .

WU Y , ABDALMAGEED W , NATARAJAN P . ManTra-Net:manipulation tracing network for detection and localization of image forgeries with anomalous features [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2019 : 9535 - 9544 .

CHEN X R , DONG C B , JI J Q , et al . Image manipulation detection by multi-view multi-scale supervision [C ] // Proceedings of International Conference on Computer Vision . 2021 .

WANG J , WU Z , CHEN J , et al . ObjectFormer for image manipulation detection and localization [C ] // Proceedings of IEEE Conference on Computer Vision and Pattern Recognition . 2022 .

DONG J , WANG W , TAN T N . CASIA image tampering detection evaluation database [C ] // Proceedings of 2013 IEEE China Summit and International Conference on Signal and Information Processing . 2013 : 422 - 426 .

HSU Y F , CHANG S F . Detecting image splicing using geometry invariants and camera characteristics consistency [C ] // Proceedings of IEEE International Conference on Multimedia and Expo . 2006 .

浏览量

134

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

DynaSynth：面向非结构化网络安全语料的大语言模型微调数据动态生成代理工具

基于数据增强的多视图对比学习图异常检测

基于操作注意力和数据增强的内部威胁检测

面向Logo识别的合成数据生成方法研究