深度学习中对抗样本的构造及防御研究

段广晗; 马春光; 宋蕾; 武朋

doi:10.11959/j.issn.2096-109x.2020016

您当前的位置：

首页 >

文章列表页 >

深度学习中对抗样本的构造及防御研究

综述 | 更新时间：2024-06-03

- 深度学习中对抗样本的构造及防御研究
- Research on structure and defense of adversarial example in deep learning
- 网络与信息安全学报 2020年6卷第2期页码：1-11
- 作者机构：
  
  1. 哈尔滨工程大学计算机科学与技术学院，黑龙江哈尔滨 150001
  2. 山东科技大学计算机科学与工程学院，山东青岛 266590
- 作者简介：
  
  [ "段广晗（1994– ），男，黑龙江海伦人，哈尔滨工程大学博士生，主要研究方向为深度学习、对抗样本、机器学习" ]
  [ "马春光（1974– ），男，黑龙江双城人，山东科技大学教授、博士生导师，主要研究方向为密码学、数据安全与隐私、人工智能安全与隐私、区块链技术与应用" ]
  [ "宋蕾（1989– ），女，黑龙江牡丹江人，哈尔滨工程大学博士生，主要研究方向为机器学习安全与隐私保护、云计算、网络安全" ]
  [ "武朋（1974– ），女，黑龙江齐齐哈尔人，山东科技大学讲师，主要研究方向为网络安全、隐私保护" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61472097);国家自然科学基金资助项目;The National Natural Science Foundation of China(61932005);国家自然科学基金资助项目;The National Natural Science Foundation of China(U1936112);黑龙江省自然科学基金资助项目;The Natural Science Foundation of Heilongjiang Province(JJ2019LH1770)
- DOI：10.11959/j.issn.2096-109x.2020016
  中图分类号： TP309
- 网络首发：2020-04，
  
  纸质出版：2020-04-15
- 稿件说明：
移动端阅览
段广晗, 马春光, 宋蕾, 等. 深度学习中对抗样本的构造及防御研究[J]. 网络与信息安全学报, 2020,6(2):1-11.

Guanghan DUAN, Chunguang MA, Lei SONG, et al. Research on structure and defense of adversarial example in deep learning[J]. Chinese Journal of Network and Information Security, 2020, 6(2): 1-11.
段广晗, 马春光, 宋蕾, 等. 深度学习中对抗样本的构造及防御研究[J]. 网络与信息安全学报, 2020,6(2):1-11. DOI： 10.11959/j.issn.2096-109x.2020016.

Guanghan DUAN, Chunguang MA, Lei SONG, et al. Research on structure and defense of adversarial example in deep learning[J]. Chinese Journal of Network and Information Security, 2020, 6(2): 1-11. DOI： 10.11959/j.issn.2096-109x.2020016.

摘要

随着深度学习技术在计算机视觉、网络安全、自然语言处理等领域的进一步发展，深度学习技术逐渐暴露了一定的安全隐患。现有的深度学习算法无法有效描述数据本质特征，导致算法面对恶意输入时可能无法给出正确结果。以当前深度学习面临的安全威胁为出发点，介绍了深度学习中的对抗样本问题，梳理了现有的对抗样本存在性解释，回顾了经典的对抗样本构造方法并对其进行了分类，简述了近年来部分对抗样本在不同场景中的应用实例，对比了若干对抗样本防御技术，最后归纳对抗样本研究领域存在的问题并对这一领域的发展趋势进行了展望。

Abstract

With the further promotion of deep learning technology in the fields of computer vision

network security and natural language processing

which has gradually exposed certain security risks.Existing deep learning algorithms can not effectively describe the essential characteristics of data or its inherent causal relationship.When the algorithm faces malicious input

it often fails to give correct judgment results.Based on the current security threats of deep learning

the adversarial example problem and its characteristics in deep learning applications were introduced

hypotheses on the existence of adversarial examples were summarized

classic adversarial example construction methods were reviewed and recent research status in different scenarios were summarized

several defense techniques in different processes were compared

and finally the development trend of adversarial example research were forecasted.

关键词

Keywords

references

SZEGEDY C , VANHOUCKE V , IOFFE S , et al . Rethinking the inception architecture for computer vision [C ] // The IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 2818 - 2826 .

TANG T A , MHAMDI L , MCLERNON D , et al . Deep learning approach for network intrusion detection in software defined networking [C ] // 2016 International Conference on Wireless Networks and Mobile Communications (WINCOM) . 2016 : 258 - 263 .

COLLOBERT R , WESTON J . A unified architecture for natural language processing:deep neural networks with multitask learning [C ] // The 25th International Conference on Machine Learning . 2008 : 160 - 167 .

CHEN C , SEFF A , KORNHAUSER A , et al . Deepdriving:learning affordance for direct perception in autonomous driving [C ] // The IEEE International Conference on Computer Vision . 2015 : 2722 - 2730 .

CHING T , HIMMELSTEIN D S , BEAULIEU-JONES B K , et al . Opportunities and obstacles for deep learning in biology and medicine [J ] . Journal of The Royal Society Interface , 2018 , 15 ( 141 ).

SZEGEDY C , ZAREMBA W , SUTSKEVER I , et al . Intriguing properties of neural networks [J ] . arXiv preprint arXiv:1312.6199 , 2013

KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial examples in the physical world [J ] . arXiv preprint arXiv:1607.02533 , 2016

ALZANTOT M , SHARMA Y , ELGOHARY A , et al . Generating natural language adversarial examples [J ] . arXiv preprint arXiv:1804.07998 , 2018

QIN Y , CARLINI N , GOODFELLOW I , et al . Imperceptible,robust,and targeted adversarial examples for automatic speech recognition [J ] . arXiv preprint arXiv:1903.10346 , 2019

LECUN Y , BENGIO Y , HINTON G . Deep learning [J ] . Nature , 2015 , 521 ( 7553 ):436.

PAPERNOT N , MCDANIEL P , GOODFELLOW I . Transferability in machine learning:from phenomena to black-box attacks using adversarial samples [J ] . arXiv preprint arXiv:1605.07277 , 2016

PAPERNOT N , MCDANIEL P , JHA S , et al . The limitations of deep learning in adversarial settings [C ] // The 1st IEEE European Symposium on Security and Privacy . 2016 .

宋蕾 , 马春光 , 段广晗 . 机器学习安全及隐私保护研究进展 [J ] . 网络与信息安全学报 , 2018 , 4 ( 8 ): 1 - 11 .

SONG L , MA C G , DUAN G H . Machine learning security and privacy:a survey [J ] . Chinese Journal of Network and Information Security , 2018 , 4 ( 8 ): 1 - 11 .

GU S , RIGAZIO L . Towards deep neural network architectures robust to adversarial examples [J ] . arXiv preprint arXiv:1412.5068 , 2014

GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [C ] // 2015 International Conference on Learning Representations . 2015 : 1 - 10 .

TABACOF P , VALLE E . Exploring the space of adversarial images [J ] . arXiv preprint arXiv:1510.05328 , 2015

TRAM`ER F , PAPERNOT N , GOODFELLOW I , et al . The space of transferable adversarial examples [J ] . arXiv preprint arXiv:1704.03453 , 2017

KROTOV D , HOPFIELD J J . Dense associative memory is robust to adversarial inputs [J ] . arXiv preprint arXiv:1701.00939 , 2017

LUO Y , BOIX X , ROIG G , et al . Foveation-based mechanisms alleviate adversarial examples [J ] . arXiv preprint arXiv:1511.06292 , 2015

TANAY T , GRIFFIN L . A boundary tilting perspective on the phenomenon of adversarial examples [J ] . arXiv preprint arXiv:1608.07690 , 2016

MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O , et al . Universal adversarial perturbations [C ] // The IEEE Conference on Computer Vision and Pattern Recognition . 2017 : 1765 - 1773 .

MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O , et al . Analysis of universal adversarial perturbations [J ] . arXiv preprint arXiv:1705.09554 , 2017

TRAM`ER F , KURAKIN A , PAPERNOT N , et al . Ensemble adversarial training:attacks and defenses [J ] . arXiv preprint arXiv:1705.07204 , 2017

MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O , et al . Robustness of classifiers to universal perturbations:a geometric perspective [C ] // International Conference on Learning Representations . 2018 .

SONG Y , KIM T , NOWOZIN S , et al . Pixeldefend:leveraging generative models to understand and defend against adversarial examples [J ] . arXiv preprint arXiv:1710.10766 , 2017

MENG D , CHEN H . Magnet:a two-pronged defense against adversarial examples [C ] // The 2017 ACM SIGSAC Conference on Computer and Communications Security . 2017 : 135 - 147 .

GHOSH P , LOSALKA A , BLACK M J . Resisting adversarial attacks using gaussian mixture variational autoencoders [J ] . arXiv preprint arXiv:1806.00081 , 2018

LEE H , HAN S , LEE J . Generative adversarial trainer:defense to adversarial perturbations with gan [J ] . arXiv preprint arXiv:1705.03387 , 2017

GILMER J , METZ L , FAGHRI F , et al . Adversarial spheres [J ] . arXiv preprint arXiv:1801.02774 , 2018

GILMER J , METZ L , FAGHRI F , et al . The relationship between high-dimensional geometry and adversarial examples [J ] . arXiv:1801.02774v3 , 2018

EYKHOLT K , EVTIMOV I , FERNANDES E , et al . Robust physical-world attacks on deep learning visual classification [C ] // The IEEE Conference on Computer Vision and Pattern Recognition . 2018 : 1625 - 1634 .

MOOSAVI-DEZFOOLI S M , FAWZI A , FROSSARD P . Deepfool:a simple and accurate method to fool deep neural networks [C ] // The 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2016 .

PAPERNOT N , MCDANIEL P , SWAMI A , et al . Crafting adversarial input sequences for recurrent neural networks [C ] // MILCOM 2016-2016 IEEE Military Communications Conference . 2016 : 49 - 54 .

GROSSE K , PAPERNOT N , MANOHARAN P , et al . Adversarial examples for malware detection [C ] // European Symposium on Research in Computer Security . 2017 : 62 - 79 .

RUSSAKOVSKY O , DENG J , SU H , et al . ImageNet large scale visual recognition challenge [J ] . International Journal of Computer Vision , 2015 , 115 ( 3 ): 211 - 252 .

PAPERNOT N , MCDANIEL P , JHA S , et al . The limitations of deep learning in adversarial settings [C ] // 2016 IEEE European Symposium on Security and Privacy . 2016 : 372 - 387 .

PAPERNOT N , MCDANIEL P , GOODFELLOW I , et al . Practical black-box attacks against machine learning [C ] // The 2017 ACM on Asia Conference on Computer and Communications Security . 2017 : 506 - 519 .

ILYAS A , ENGSTROM L , ATHALYE A , et al . Black-box adversarial attacks with limited queries and information [J ] . arXiv preprint arXiv:1804.08598 , 2018

BALUJA S , FISCHER I . Adversarial transformation networks:Learning to generate adversarial examples [J ] . arXiv preprint arXiv:1703.09387 , 2017

XIAO C , LI B , ZHU J Y , et al . Generating adversarial examples with adversarial networks [C ] // The 27th International Joint on Artificial Intelligence Main track . 2019 : 3805 - 3911 .

ZHAO P , FU Z , HU Q , et al . Detecting adversarial examples via key-based network [J ] . arXiv preprint arXiv:1806.00580 , 2018

MENG D , CHEN H . Magnet:a two-pronged defense against adversarial examples [C ] // The 2017 ACM SIGSAC Conference on Computer and Communications Security . 2017 : 135 - 147 .

XU W , EVANS D , QI Y . Feature squeezing:detecting adversarial examples in deep neural networks [J ] . arXiv preprint arXiv:1704.01155 , 2017

HOSSEIN H , CHEN Y , KANNAN S , et al . Blocking transferability of adversarial examples in black-box learning systems [J ] . arXiv:1703.04318 , 2017

SABOUR S , NICHOLAS F , HINTON G E . Dynamic routing between capsules [C ] // Neural Information Processing Systems . 2017 .

NICHOLAS F , SABOUR S , HINTON G . DARCCC:detecting adversaries by reconstruction from class conditional capsules [J ] . arXiv preprint arXiv:1811.06969 , 2018

TRAMÈR F , KURAKIN A , PAPERNOT N , et al . Ensemble adversarial raining:attacks and defenses [J ] . arXiv:1705.07204 , 2017

SINHA A , CHEN Z , BADRINARAYANAN V , et al . Gradient adversarial training of neural networks [J ] . arXiv preprint arXiv:1806.08028 , 2018

KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial machine learning at scale [J ] . arXiv preprint arXiv:1611.01236 , 2016

PAPERNOT N , MCDANIEL P , WU X , et al . Distillation as a defense to adversarial perturbations against deep neural networks [C ] // 2016 IEEE Symposium on Security and Privacy . 2016 : 582 - 597 .

HINTON G E , VINYALS O , DEAN J . Distilling the knowledge in a neural network [J ] . arXiv:1503.02531 ,

LEE H , HAN S , LEE J . Generative adversarial trainer:defense to adversarial perturbations with GAN [J ] . arXiv preprint arXiv:1705.03387 , 2017

浏览量

1886

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于光学的物理域对抗攻防综述

基于图像重构的MNIST对抗样本防御算法

基于双路特征提取和自适应融合的钓鱼网站检测

DNS异常检测：大规模实验分析

FL-IoV：基于联邦学习的综合车联网入侵检测方法