Research progress of digital image forensic techniques based on deep learning

Tong QIAO; Hongwei YAO; Binmin PAN; Ming XU; Yanli CHEN

doi:10.11959/j.issn.2096-109x.2021047

您当前的位置：

首页 >

文章列表页 >

Research progress of digital image forensic techniques based on deep learning

TopicⅠ: Voice Image and Audio-Video Processing | 更新时间：2024-06-03

- Research progress of digital image forensic techniques based on deep learning
- Chinese Journal of Network and Information Security Vol. 7, Issue 5, Pages: 13-28(2021)
- 作者机构：
  
  1. 杭州电子科技大学网络空间安全学院，浙江杭州 310018
  2. 法国特鲁瓦工程技术大学，特鲁瓦 10000
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2021047
  CLC： TP37
- Online First：2021-10，
  
  Published：15 October 2021
- 稿件说明：
移动端阅览
Tong QIAO, Hongwei YAO, Binmin PAN, et al. Research progress of digital image forensic techniques based on deep learning[J]. Chinese Journal of Network and Information Security, 2021, 7(5): 13-28.
DOI：

Tong QIAO, Hongwei YAO, Binmin PAN, et al. Research progress of digital image forensic techniques based on deep learning[J]. Chinese Journal of Network and Information Security, 2021, 7(5): 13-28. DOI： 10.11959/j.issn.2096-109x.2021047.

摘要

随着数字图像篡改技术不断的革新换代，传统的取证方法已经无法对抗最新的多媒体篡改手段和技术，尤其是深度造假及深度学习技术带来的全新挑战。总结提炼了包括图像预处理模块、特征提取模块及分类结果后处理模块的通用数字图像取证框架，并在提出的框架基础之上分析现有相关研究的优缺点，同时归纳了数字图像取证面临的挑战并指明未来的发展方向。

Abstract

In the new era of rapid development of internet

where massive forgery images with updated tampering techniques flood into

traditional algorithms are no longer able to deal with the latest multimedia tampering techniques

especially those caused by Deepfake and deep learning techniques.Thus

a universal framework for image forensics including image pre-processing module

feature extraction module and post-processing module designed for specific classification were proposed creatively.Accordingly

the state-of-the-art algorithms were reviewed，and meanwhile the main strength and limitations of current algorithms were generalized.More importantly

the future studies were also listed for advancing the development of digital image forensics.

关键词

Keywords

references

梁瑞刚 , 吕培卓 , 赵月 , 等 . 视听觉深度伪造检测技术研究综述 [J ] . 信息安全学报 , 2020 , 5 ( 2 ): 1 - 17 .

LIANG R G , LYU P Z , ZHAO Y , et al . A survey of audiovisual Deepfake detection techniques [J ] . Journal of Cyber Security , 2020 , 5 ( 2 ): 1 - 17 .

国家互联网信息办公室 . 网络信息内容生态治理规定 [R ] . 2019 .

Cyberspace Administration of China . Regulations on the Governance of Network Information Content [R ] . 2019 .

QIAO T , LUO X , WU T , et al . Adaptive steganalysis based on statistical model of quantized DCT coefficients for JPEG images [J ] . IEEE Transactions on Dependable and Secure Computing , 2019 ( 99 ): 1 - 16 .

QIAO T , RETRAINT F , COGRANNE R , et al . Steganalysis of JSteg algorithm using hypothesis testing theory [J ] . EURASIP Journal on Information Security , 2015 ( 1 ): 1 - 16 .

USMAN B , DUFOUR N , SAENKO K , et al . PuppetGAN:cross-domain image manipulation by demonstra-tion [C ] // Proceedings of the IEEE International Conference on Computer Vision (ICCV) . 2019 : 9450 - 9458 .

LIANG X , ZHANG H , LIN L , et al . Generative semantic manipulation with mask-contrasting GAN [C ] // Proceedings of the European Conference on Computer Vision (ECCV) . 2018 : 558 - 573 .

PENG F , YIN L P , ZHANG L B , et al . CGR-GAN:CG facial image regeneration for anti-forensics based on generative adversarial network [J ] . IEEE Transactions on Multimedia , 2020 ( 3 ): 1 - 14 .

CHEN C , ZHAO X , STAMM M C . Generative adversarial attacks against deep-learning-based camera model identification [J ] . IEEE Transactions on Information Forensics and Security , 2019 ( 99 ): 1 - 16 .

PENG P , NING P , REEVES D S . On the secrecy of tim-ing-based active watermarking trace-back techniques [C ] // IEEE Symposium on Security and Privacy (S＆P) . 2006 : 1 - 15 .

ZHOU G , LYU D . An overview of digital watermarking in image forensics [C ] // IEEE International Joint Conference on Computational Sciences and Optimization (CSO) . 2011 : 332 - 335 .

ZHAO J , WANG Q , GUO J , et al . An overview on passive image forensics technology for automatic computer forgery [J ] . International Journal of Digital Crime and Forensics (IJDCF) , 2016 , 8 ( 4 ): 14 - 25 .

SONI B , DAS P K , THOUNAOJAM D M . Copy-Move tampering detection based on local binary pattern histogram fourier feature [C ] // Proceedings of the 7th International Conference on Computer and Communication Technology . 2017 : 78 - 83 .

YAO H , WANG S , ZHANG X , et al . Detecting image splicing based on noise level inconsistency [J ] . Multimedia Tools and Applications , 2017 , 76 ( 10 ): 12457 - 12479 .

JIN X , SU Y , ZOU L , et al . Video logo removal detection based on sparse representation [J ] . Multimedia Tools and Applications , 2018 , 77 ( 22 ): 29303 - 29322 .

CHEN J , KANG X , LIU Y . Median filtering forensics based on convolutional neural networks [J ] . IEEE Signal Processing Letters , 2015 , 22 ( 11 ): 1849 - 1853 .

QIAO T , ZHU A , RETRAINT F . Exposing image resampling forgery by using linear parametric model [J ] . Multimedia Tools and Applications , 2018 , 77 ( 2 ): 1501 - 1523 .

QIAO T , SHI R , LUO X , et al . Statistical model-based detector via texture weight map:application in re-sampling authentication [J ] . IEEE Transactions on Multimedia , 2019 , 21 ( 5 ): 1077 - 1092 .

THAI T H , COGRANNE R , RETRAINTETAL F . JPEG quantization step estimation and its applications to digital image forensics [J ] . IEEE Transactions on Information Forensics and Security , 2017 , 12 ( 1 ): 123 - 133 .

QIAN Y , DONG J , WANG W , et al . Deep learning for steganalysis via convolutional neural networks [J ] . Media Watermarking,Security,and Forensics , 2015 ( 94091 ): 94090J1 - 94090J10 .

BAYAR B , STAMM M C . Constrained convolutional neural networks:a new approach towards general purpose image manipulation detection [J ] . IEEE Transactions on Information Forensics and Security , 2018 , 13 ( 11 ): 2691 - 2706 .

QUAN W , WANG K , YAN D M . Distinguishing between natural and computer-generated images using convolutional neural networks [J ] . IEEE Transactions on Information Forensics and Security , 2018 , 13 ( 11 ): 2772 - 2787 .

BAYAR B , STAMM M C . Augmented convolutional feature maps for robust CNN-based camera model identifica-tion [C ] // 2017 IEEE International Conference on Image Processing (ICIP) . 2017 : 4098 - 4102 .

FRIDRICH J , KODOVSKY J . Rich models for steganalysis of digital images [J ] . IEEE Transactions on Information Forensics and Security , 2012 , 7 ( 3 ): 868 - 882 .

BAYAR B , STAMM M C . A generic approach towards image manipulation parameter estimation using convolutional neural networks [C ] // Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security (IH＆MMSec) . 2017 : 147 - 157 .

BAYAR B , STAMMM C . A deep learning approach to universal image manipulation detection using a new convolutional layer [C ] // Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security (IH＆MMSec) . 2016 : 5 - 10 .

BAYRAM S , SENCAR H , MEMON N , et al . Source camera identification based on CFA interpolation [C ] // IEEE International Conference on Image Processing (ICIP) . 2005 : 1 - 4 .

CAO H , KOTA C . Accurate detection of demosaicing regularity for digital image forensics [J ] . IEEE Transactions on Information Forensics and Security , 2009 , 4 ( 4 ): 899 - 910 .

FAN J , CHEN T , KOT A C . Exif-white balance recognition for image forensic analysis [J ] . Multidimensional Systems and Signal Processing , 2017 , 28 ( 3 ): 795 - 815 .

LUKÁŠ J , FRIDRICH J , GOLJAN M . Digital camera identification from sensor pattern noise [J ] . IEEE Transactions on Information Forensics and Security , 2006 , 1 ( 2 ): 205 - 214 .

FILLER T , FRIDRICH J , GOLJAN M . Using sensor pattern noise for camera model identification [C ] // IEEE International Conference on Image Processing(ICIP) . 2008 : 1296 - 1299 .

MARRA F , POGGI G , SANSONE C , et al . Blind PRNU-based image clustering for source identification [J ] . IEEE Transactions on Information Forensics and Security , 2017 , 12 ( 9 ): 2197 - 2211 .

COZZOLINO D , MARRA F , GRAGANIEEL O D , et al . Combining PRNU and noiseprint for robust and efficient device source identification [J ] . EURASIP Journal on Information Security , 2020 , 2020 ( 1 ): 1 - 12 .

GOLJAN M , FRIDRICH J , KIRCHNER M . Image manipulation detection using sensor linear pattern [J ] . Electronic Imaging , 2018 ( 7 ).

LAWGALY A , KHELIFI F . Sensor pattern noise estimation based on improved locally adaptive DCT filtering and weighted averaging for source camera identification and verification [J ] . IEEE Transactions on Information Forensics and Security , 2016 , 12 ( 2 ): 92 - 404 .

VALSESIA D , COLUCCIA G , BIANCHI T , et al . User authentication via PRNU-based physical unclonable functions [J ] . IEEE Transactions on Information Forensics and Security . 2017 , 12 ( 8 ): 1941 - 1956 .

TUAMA A , COMBY F , CHAUMONT M . Camera model identification with the use of deep convolutional neural networks [C ] // IEEE Int Workshop Information Forensics Security (WIFS) . 2016 : 1 - 6 .

BONDI L , BAROFFIO L , GÜERA D , , et al . First steps toward camera model identification with Convolutional Neural Networks [J ] . IEEE Signal Processing Letters , 2017 , 24 ( 3 ): 259 - 263 .

BONDI L , LAMERI S , GÜERA D ,, et al . Tampering detection and localization through clustering of camera-based CNN features [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . 2017 : 1855 - 1864 .

LESHNO M , LINV Y , PINKUS A . Multilayer feedforward networks with a non-polynomial activation function can approximate any function [J ] . Neural Networks , 1993 , 6 ( 6 ): 861 - 867 .

YAO H , QIAO T , XU M , et al . Robust multi-classifier for camera model identification based on convolution neural network [J ] . IEEE Access , 2018 ( 6 ): 24973 - 24982 .

CHEN Y , KANG X , SHIY Q , et al . A multi-purpose image forensic method using densely connected convolutional neural networks [J ] . Journal of Real-Time Image Processing , 2019 , 16 ( 3 ): 725 - 740 .

HUANG G , LIU Z , VAN DE R MAATEN L , et al . Densely connected convolutional networks [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2017 : 4700 - 4708 .

LIU Y , GUAN Q , ZHAO X , et al . Image forgery localization based on multi-scale convolutional neural net-works [C ] // Proceedings of the 6th ACM Workshop on Information Hiding and Multimedia Security (IH＆MMSec) . 2018 : 85 - 90 .

WU Y , ABD-ALMAGEED W , NATARAJAN P . Mantra-net:Manipulation tracing network for detection and localization of image forgeries with anomalous features [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2019 : 9543 - 9552 .

KORUS P , HUANG J . Multi-scale fusion for improved localization of malicious tampering in digital images [J ] . IEEE Transactions on Image Processing , 2016 , 25 ( 3 ): 1312 - 1326 .

DENG C , LI Z , GAO X , et al . Deep multi-scale discriminative networks for double JPEG compression forensics [J ] . ACM Transactions on Intelligent Systems and Technology , 2019 , 10 ( 2 ): 1 - 20 .

LI H , LUO W , QIU X , et al . Image forgery localization via integrating tampering possibility maps [J ] . IEEE Transactions on Information Forensics and Security , 2017 , 12 ( 5 ): 1240 - 1252 .

LI H , LUO W , QIU X , et al . Identification of various image operations using residual-based features [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2016 , 28 ( 1 ): 31 - 45 .

BARNES C , SHECHTMAN E , FINKELSTEIN A , et al . Patchmatch:a randomized correspondence algorithm for structural image editing [J ] . ACM Transactions on Graphics (ToG) , 2009 , 28 ( 3 ): 1 - 11 .

NATARAJ L , MOHAMMED T M , CHANDRASEKARANS , et al . Detecting GAN generated fake images using co-occurrence matrices [J ] . Electronic Imaging , 2019 ,5.

MCCLOSKEY S , ALBRIGHT M . Detecting GAN-Generated Imagery Using Saturation Cues [C ] // IEEE International Conference on Image Processing (ICIP) . 2019 : 4584 - 4588 .

MATERN F , RIESS C , STAMMINGER M . Exploiting visual artifacts to expose Deepfakes and face manipulations [C ] // IEEE Winter Applications of Computer Vision Workshops (WACVW) . 2019 : 83 - 92 .

MO H , CHEN B , LUO W . Fake faces identification via convolutional neural network [C ] // Proceedings of the 6th ACM Workshop on Information Hiding and Multimedia Security (IH＆MMSec) . 2018 : 43 - 47 .

AFCHAR D , NOZICK V , YAMAGISHI J , et al . MesoNet:a compact facial video forgery detection network [C ] // IEEE International Workshop on Information Forensics and Security (WIFS) . 2018 : 1 - 7 .

THIES J , ZOLLHOFER M , STAMMINGER M , et al . Face2face:real-time face capture and reenactment of RGB videos [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2016 : 2387 - 2395 .

LI Y , CHANGM C , LYU S . In ICTU oculi:exposing AI generated fake face videos by detecting eye blinking [C ] // In IEEE International Workshop on Information Forensics and Security (WIFS) . 2018 : 1 - 7 .

YANG X , LI Y , LYU S . Exposing deep fakes using inconsistent head poses [C ] // IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP) . 2019 : 8261 - 8265 .

AGARWAL S , FARID H , GU Y , et al . Protecting world leaders against Deepfakes [C ] // IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . 2019 : 38 - 45 .

ZHOU P , HAN X , MORARIU V I , et al . Two-stream neural networks for tampered face detection [C ] // IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . 2017 :839.

NGUYEN H H , YAMAGISHI J , ECHIZEN I . Capsule-forensics:Using capsule networks to detect forged images and videos [C ] // IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP) . 2019 : 2307 - 2311 .

ROSSLER A , COZZOLINO D , VERDOLIVA L , et al . FaceForensics++:learning to detect manipulated facial images [C ] // IEEE International Conference on Computer Vision (ICCV) . 2019 : 1 - 11 .

KOOPMAN M , RODRIGUEZ A M , GERADTS Z . Detection of Deepfake video manipulation [C ] // IMVIP . 2018 : 133 - 136 .

AMERINI I , GALTERI L , CALDELLI R , et al . Deepfake video detection through optical flow basedCNN [C ] // Proceedings of the IEEE International Conference on Computer Vision Workshops . 2019 : 1 - 3 .

LI Y , LYU S . Exposing Deepfake videos by detecting face warping artifacts [C ] // IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . 2019 : 46 - 52 .

DURALL R , KEUPER M , KEUPER J . Watch your up-convolution:CNN based generative deep neural networks are failing to reproduce spectral distributions [C ] // IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . 2020 : 7890 - 7899 .

LI L , BAO J , ZHANG T , et al . Face X-ray for more general face forgery detection [C ] // IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . 2020 : 5001 - 5010 .

LIUZ , QIX , TORRP H , . Global texture enhancement for fake face detection in the wild [C ] // IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . 2020 : 8060 - 8069 .

AMERINII , LIC T , CALDELLI R . Social network identification through image classification with CNN [J ] . IEEE Access , 2019 , 7 : 35264 - 35273 .

JÚNIORP R M , BONDI L , BESTAGINI P , et al . A PRNU-based method to expose video device compositions in open-set setups [C ] // 2019 IEEE International Conference on Image Processing (ICIP) . 2019 : 96 - 100 .

HE P , JIANG X , SUN T , et al . Computer graphics identification combining convolutional and recurrent neural networks [J ] . IEEE Signal Processing Letters , 2018 , 25 ( 9 ): 1369 - 1373 .

QIAO T , RETRAINT F , COGRANNE R , et al . Individual camera device identification from JPEG images [J ] . Signal Processing:Image Communication , 2017 , 52 : 74 - 86 .

GLOE T , BÖHME R , . The dresden image database for benchmarking digital image forensics [C ] // Proceedings of the 2010 ACM Symposium on Applied Computing . 2010 : 1584 - 1590 .

HUH M , LIU A , OWENS A , et al . Fighting fake news:image splice detection via learned self-consistency [C ] // Proceedings of the European Conference on Computer Vision (ECCV) . 2018 : 101 - 117 .

BUNK J , BAPPYJ H , MOHAMMEDT M , et al . Detection and localization of image forgeries using resampling features and deep learning [C ] // IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . 2017 : 1881 - 1889 .

HUANG H , GUO W , ZHANG Y . Detection of copy-move forgery in digital images using sift algorithm [C ] // 2008 IEEE Pacific-Asia Workshop on Computational Intelligence and Industrial Application . 2008 : 272 - 276 .

LI L , LI S , ZHU H , et al . An efficient scheme for detecting copy-move forged images by local binary patterns [J ] . Journal of Information Hiding and Multimedia Signal Processing (IIH-MSP) , 2013 , 4 ( 1 ): 46 - 56 .

GAO S , LIAO X , LIU X . Real-time detecting one specific tampering operation in multiple operator chains [J ] . Journal of Real-Time Image Processing , 2019 , 16 ( 3 ): 741 - 750 .

QIAO T , RETRAINT F . Identifying individual camera device from raw images [J ] . IEEE Access , 2018 , 6 : 78038 - 78054 .

WU Y , ABD-ALMAGEED W ,, NATARAJAN P . Deep matching and validation network:an end-to-end solution to constrained image splicing localization and detection [C ] // Proceedings of the 25th ACM International Conference on Multimedia . 2017 : 1480 - 1502 .

BONDI L , BESTAGINI P , PEREZ-GONZALEZ F , , et al . Improving PRNU compression through preprocessing,quantization,and coding [J ] . IEEE Transactions on Information Forensics and Security , 2018 , 14 ( 3 ): 608 - 620 .

MOHAMMEDT M , BUNK J , NATARAJ L , et al . Boosting image forgery detection using resampling features and copy-move analysis [J ] . Electronic Imaging , 2018 ( 7 ): 1 - 7 .

MARRA F , GRAGNANIELLO D , VERDOLIVA L , et al . A full-image full-resolution end-to-end trainable CNN framework for image forgery detection [J ] . arXiv preprint arXiv:1909.06751 , 2019 .

BI X , WEI Y , XIAO B , et al . RRU-Net:the ringed residual u-net for image splicing forgery detection [C ] // IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . 2019 : 1 - 10 .

MANDELLI S , COZZOLINO D , BESTAGINI P , et al . CNN-based fast source device identification [J ] . arXiv preprint arXiv:2001.11847 , 2020 .

COZZOLINO D , MARRA F , GRAGNANIELLO D , et al . Combining PRNU and noise print for robust and efficient device source identification [J ] . EURASIP Journal on Information Security , 2020 ( 1 ): 1 - 12 .

Views

1349

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Encrypted traffic classification method based on convolutional neural network

Image recoloring detection based on inter-channel correlation

Image generation classification method based on convolution neural network

Related Author

Rongna XIE

Zhuhong MA

Zongyu LI

Ye TIAN

Nuo CHEN

Shuren QI

Yushu ZHANG

Mingfu XUE

Related Institution

College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics

College of Computer Science and Technology, Harbin Institute of Technology

State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences

University of Chinese Academy of Sciences

⁰