基于双分支网络的图像修复取证方法

章登勇; 文凰; 李峰; 曹鹏; 向凌云; 杨高波; 丁湘陵

doi:10.11959/j.issn.2096-109x.2022084

您当前的位置：

首页 >

文章列表页 >

基于双分支网络的图像修复取证方法

学术论文 | 更新时间：2024-06-03

- 基于双分支网络的图像修复取证方法
- Image inpainting forensics method based on dual branch network
- 网络与信息安全学报 2022年8卷第6期页码：110-122
- 作者机构：
  
  1. 长沙理工大学湖南省综合交通运输大数据智能处理重点实验室，湖南长沙 410114
  2. 长沙理工大学计算机与通信工程学院，湖南长沙 410114
  3. 湖南大学信息科学与工程学院，湖南长沙 410082
  4. 湖南科技大学计算机科学与工程学院，湖南湘潭 411004
- 作者简介：
  
  [ "章登勇（1980- ），男，江西南昌人，长沙理工大学副教授，主要研究方向为多媒体信息安全、图像处理与模式识别" ]
  [ "文凰（1996- ），女，湖南望城人，长沙理工大学硕士生，主要研究方向为多媒体信息安全" ]
  [ "李峰（1964- ），男，湖南常德人，长沙理工大学教授，主要研究方向为多媒体信息安全、图像处理与模式识别" ]
  [ "曹鹏（1982- ），男，湖南常德人，长沙理工大学讲师，主要研究方向为图像处理与模式识别、图像取证" ]
  [ "向凌云（1983- ），女，湖南娄底人，长沙理工大学副教授，主要研究方向为信息安全、信息隐藏与数字水印、隐写分析、自然语言处理、模式识别与机器学习" ]
  [ "杨高波（1973- ），湖南岳阳人，湖南大学教授、博士生导师，主要研究方向为图像/视频信息安全、多媒体通信、纹理压缩" ]
  [ "丁湘陵（1981- ），男，湖南株洲人，博士，湖南科技大学副教授，主要研究方向为多媒体内容安全、图像/视频处理、密码分析" ]
- 基金信息：
  
  国家自然科学基金;The National Natural Science Foundation of China(62172059);国家自然科学基金;The National Natural Science Foundation of China(61972057);国家自然科学基金;The National Natural Science Foundation of China(62072055);湖南省自然科学基金;Natural Science Foundation of Hunan Province(2020JJ4626);湖南省自然科学基金;Natural Science Foundation of Hunan Province(2020JJ4029);湖南省教育厅优秀青年项目;Scientific Research Fund of Hunan Provincial Education Department of China(19B004)
- DOI：10.11959/j.issn.2096-109x.2022084
  中图分类号： TP393
- 网络首发：2022-12，
  
  纸质出版：2022-12-15
- 稿件说明：
移动端阅览
章登勇, 文凰, 李峰, 等. 基于双分支网络的图像修复取证方法[J]. 网络与信息安全学报, 2022,8(6):110-122.

Dengyong ZHANG, Huang WEN, Feng LI, et al. Image inpainting forensics method based on dual branch network[J]. Chinese Journal of Network and Information Security, 2022, 8(6): 110-122.
章登勇, 文凰, 李峰, 等. 基于双分支网络的图像修复取证方法[J]. 网络与信息安全学报, 2022,8(6):110-122. DOI： 10.11959/j.issn.2096-109x.2022084.

Dengyong ZHANG, Huang WEN, Feng LI, et al. Image inpainting forensics method based on dual branch network[J]. Chinese Journal of Network and Information Security, 2022, 8(6): 110-122. DOI： 10.11959/j.issn.2096-109x.2022084.

摘要

图像修复是一项利用图像已知区域的信息来修复图像中缺失或损坏区域的技术。人们借助以此为基础的图像编辑软件无须任何专业基础就可以轻松地编辑和修改数字图像内容，一旦图像修复技术被用于恶意移除图像的内容，会给真实的图像带来信任危机。目前图像修复取证的研究只能有效地检测某一种类型的图像修复。针对这一问题，提出了一种基于双分支网络的图像修复被动取证方法。双分支中的高通滤波卷积网络先使用一组高通滤波器来削弱图像中的低频分量，然后使用4个残差块提取特征，再进行两次4倍上采样的转置卷积对特征图进行放大，此后使用一个 5×5 的卷积来减弱转置卷积带来的棋盘伪影，生成图像高频分量上的鉴别特征图。双分支中的双注意力特征融合分支先使用预处理模块为图像增添局部二值模式特征图。然后使用双注意力卷积块自适应地集成图像局部特征和全局依赖，捕获图像修复区域和原始区域在内容及纹理上的差异，再对双注意力卷积块提取的特征进行融合。最后对特征图进行相同的上采样，生成图像内容和纹理上的鉴别特征图。实验结果表明该方法在检测移除对象的修复区域上，针对样本块修复方法上检测的F1分数较排名第二的方法提高了2.05%，交并比上提高了3.53%；针对深度学习修复方法上检测的F1分数较排名第二的方法提高了1.06%，交并比提高了1.22%。对结果进行可视化可以看出，在检测修复区域上能够准确地定位移除对象的边缘。

Abstract

Image inpainting is a technique that uses information from known areas of an image to repair missing or damaged areas of the image.Image editing software based on it has made it easy to edit and modify the content of digital images without any specialized foundation.When image inpainting techniques are used to maliciously remove the content of an image

it will cause confidence crisis on the real image.Current researches in image inpainting forensics can only effectively detect a certain type of image inpainting.To address this problem

a passive forensic method for image inpainting was proposed

which is based on a two-branch network.The high-pass filtered convolutional network in the dual branch first used a set of high-pass filters to attenuate the low-frequency components in the image.Then features were extracted using four residual blocks

and two transposed convolutions were performed with 4x up-sampling to zoom in on the feature map.And thereafter a 5×5 convolution was used to attenuate the tessellation artifacts from the transposed convolutions to generate a discriminative feature map on the high-frequency components of the image.The dual-attention feature fusion branch in the dual branch first added a local binary pattern feature map to the image using a preprocessing block.Then the dual-attention convolution block was used to adaptively integrate the image’s local features and global dependencies to capture the differences in content and texture between the inpainted and pristine regions of the image.Additionally

the features extracted from the dual-attention convolution block were fused

and the feature maps were up-sampled identically to generate the discriminative image content and texture on the feature maps.The extensive experimental results show the proposed method improved the F1 score by 2.05% and the Intersection over Union(IoU) by 3.53% for the exemplar-based method and by 1.06% and 1.22% for the deep-learning-based method in detecting the inpainted region of the removed object.Visualization of the results shows that the edges of the removed objects can be accurately located on the detected inpainted area.

关键词

Keywords

references

SRIDEVI G , SRINIVAS KUMAR S . Image inpainting based on fractional-order nonlinear diffusion for image reconstruction [J ] . Circuits,Systems,and Signal Processing , 2019 , 38 ( 8 ): 3802 - 3817 .

DOU L , QIAN Z , QIN C , et al . Anti-forensics of diffusion-based image inpainting [J ] . Journal of Electronic Imaging , 2020 , 29 ( 4 ): 043026 .

ZHANG Y , DING F , KWONG S , et al . Feature pyramid network for diffusion-based image inpainting detection [J ] . Information Sciences , 2021 , 572 : 29 - 42 .

CRIMINISI A , PéREZ P , TOYAMA K . Region filling and object removal by exemplar-based image inpainting [J ] . IEEE Transactions on image processing , 2004 , 13 ( 9 ): 1200 - 1212 .

QUY TRAN B , VAN NGUYEN T , DUY TRAN D , et al . Accelerating exemplar-based image inpainting with GPU and CUDA [C ] // 2021 10th International Conference on Software and Computer Applications . 2021 : 173 - 179 .

XU T , HUANG T Z , DENG L J , et al . Exemplar-based image inpainting using adaptive two-stage structure-tensor based priority function and nonlocal filtering [J ] . Journal of Visual Communication and Image Representation , 2022 ,83:103430.

IIZUKA S , SIMO-SERRA E , ISHIKAWA H . Globally and locally consistent image completion [J ] . ACM Transactions on Graphics (ToG) , 2017 , 36 ( 4 ): 1 - 14 .

ZENG Y , FU J , CHAO H , et al . Learning pyramid-context encoder network for high-quality image inpainting [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2019 : 1486 - 1494 .

SUVOROV R , LOGACHEVA E , MASHIKHIN A , et al . Resolution-robust large mask inpainting with fourier convolutions [C ] // Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision . 2022 : 2149 - 2159 .

WU Q , SUN S-J , ZHU W , et al . Detection of digital doctoring in exemplar-based inpainted images [C ] // 2008 International Conference on Machine Learning and Cybernetics . 2008 , 3 : 1222 - 1226 .

BACCHUWAR K S , RAMAKRISHNAN K . A jump patch-block match algorithm for multiple forgery detection [C ] // 2013 International Mutli-Conference on Automation,Computing,Communication,Control and Compressed Sensing (iMac4s) . 2013 : 723 - 728 .

CHANG I C , YU J C , CHANG C C . A forgery detection algorithm for exemplar-based inpainting images using multi-region relation [J ] . Image and Vision Computing , 2013 , 31 ( 1 ): 57 - 71 .

LIANG Z , YANG G , DING X , et al . An efficient forgery detection algorithm for object removal by exemplar-based image inpainting [J ] . Journal of Visual Communication and Image Representation , 2015 , 30 : 75 - 85 .

ZHAO Y Q , LIAO M , SHIH F Y , et al . Tampered region detection of inpainting JPEG images [J ] . Optik , 2013 , 124 ( 16 ): 2487 - 2492 .

LIU Q , SUNG A H , ZHOU B , et al . Exposing inpainting forgery in jpeg images under recompression attacks [C ] // 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA) . 2016 : 164 - 169 .

ZHANG D , LIANG Z , YANG G , et al . A robust forgery detection algorithm for object removal by exemplar-based image inpainting [J ] . Multimedia Tools and Applications , 2018 , 77 ( 10 ): 11823 - 11842 .

ZHU X , QIAN Y , ZHAO X , et al . A deep learning approach to patch-based image inpainting forensics [J ] . Signal Processing:Image Communication , 2018 , 67 : 90 - 99 .

WANG X , WANG H , NIU S . An intelligent forensics approach for detecting patch-based image inpainting [J ] . Mathematical Problems in Engineering , 2020 , 2020 ( 8 ): 1 - 10 .

WANG X , NIU S , WANG H . Image inpainting detection based on multi-task deep learning network [J ] . IETE Technical Review , 2021 , 38 ( 1 ): 149 - 157 .

LU M , NIU S . A detection approach using LSTM-CNN for object removal caused by exemplar-based image inpainting [J ] . Electronics , 2020 , 9 ( 5 ): 858 .

BARNI M , COSTANZO A , NOWROOZI E , et al . CNN-based detection of generic contrast adjustment with JPEG post-processing [C ] // 2018 25th IEEE International Conference on Image Processing (ICIP) . 2018 : 3803 - 3807 .

SHAN W , YI Y , HUANG R , et al . Robust contrast enhancement forensics based on convolutional neural networks [J ] . Signal Processing:Image Communication , 2019 , 71 : 138 - 146 .

ZHOU P , HAN X , MORARIU V I , et al . Learning rich features for image manipulation detection [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 : 1053 - 1061 .

BARNI M , BONDI L , BONETTINI N , et al . Aligned and non-aligned double JPEG detection using convolutional neural networks [J ] . Journal of Visual Communication and Image Representation , 2017 , 49 : 153 - 163 .

BAYAR B , STAMM M C . Constrained convolutional neural networks:a new approach towards general purpose image manipulation detection [J ] . IEEE Transactions on Information Forensics and Security , 2018 , 13 ( 11 ): 2691 - 2706 .

BUNK J , BAPPY J H , MOHAMMED T M , et al . Detection and localization of image forgeries using resampling features and deep learning [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . 2017 : 1881 - 1889 .

BAPPY J H , SIMONS C , NATARAJ L , et al . Hybrid lstm and encoder–decoder architecture for detection of image forgeries [J ] . IEEE Transactions on Image Processing , 2019 , 28 ( 7 ): 3286 - 3300 .

WU Y , ABDALMAGEED W , NATARAJAN P . Mantra-net:Manipulation tracing network for detection and localization of image forgeries with anomalous features [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2019 : 9543 - 9552 .

BARNI M , PHAN Q T , TONDI B . Copy move source-target disambiguation through multi-branch CNNs [J ] . IEEE Transactions on Information Forensics and Security , 2020 , 16 : 1825 - 1840 .

LI H , HUANG J . Localization of deep inpainting using high-pass fully convolutional network [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . 2019 : 8301 - 8310 .

WANG X , WANG H , NIU S . An image forensic method for AI inpainting using faster R-CNN [C ] // International Conference on Artificial Intelligence and Security . 2019 , 41 : 476 - 487 .

LI A , KE Q , MA X , et al . Noise doesn't lie:towards universal detection of deep inpainting [C ] // Proceedings of International Joint Conference on Artificial Intelligence . 2021 : 786 - 792 .

WU H , ZHOU J . IID-Net:image inpainting detection network via neural architecture search and attention [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2021 , 32 ( 3 ): 1172 - 1185 .

SZEGEDY C , LIU W , JIA Y , et al . Going deeper with convolutions [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2015 : 1 - 9 .

ZHANG D , CHEN X , LI F , et al . Seam-carved image tampering detection based on the cooccurrence of adjacent lbps [J ] . Security and Communication Networks , 2020 , 2020 : 1 - 12 .

FU J , LIU J , TIAN H , et al . Dual attention network for scene segmentation [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2019 : 3146 - 3154 .

EVERINGHAM M , ESLAMI S , VAN GOOL L , et al . The pascal visual object classes challenge:a retrospective [J ] . International Journal of Computer Vision , 2015 , 111 ( 1 ): 98 - 136 .

浏览量

286

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于程序过程间语义优化的深度学习漏洞检测方法

融合色彩空间特征与频域特征的多源伪造图像检测方法

基于双路特征提取和自适应融合的钓鱼网站检测

DNS异常检测：大规模实验分析

FL-IoV：基于联邦学习的综合车联网入侵检测方法