Research on structure and defense of adversarial example in deep learning

Guanghan DUAN; Chunguang MA; Lei SONG; Peng WU

doi:10.11959/j.issn.2096-109x.2020016

您当前的位置：

首页 >

文章列表页 >

Research on structure and defense of adversarial example in deep learning

Comprehensive Reviews | 更新时间：2024-06-03

- Research on structure and defense of adversarial example in deep learning
- Chinese Journal of Network and Information Security Vol. 6, Issue 2, Pages: 1-11(2020)
- 作者机构：
  
  1. 哈尔滨工程大学计算机科学与技术学院，黑龙江哈尔滨 150001
  2. 山东科技大学计算机科学与工程学院，山东青岛 266590
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2020016
  CLC： TP309
- Online First：2020-04，
  
  Published：15 April 2020
- 稿件说明：
移动端阅览
Guanghan DUAN, Chunguang MA, Lei SONG, et al. Research on structure and defense of adversarial example in deep learning[J]. Chinese Journal of Network and Information Security, 2020, 6(2): 1-11.
DOI：

Guanghan DUAN, Chunguang MA, Lei SONG, et al. Research on structure and defense of adversarial example in deep learning[J]. Chinese Journal of Network and Information Security, 2020, 6(2): 1-11. DOI： 10.11959/j.issn.2096-109x.2020016.

摘要

随着深度学习技术在计算机视觉、网络安全、自然语言处理等领域的进一步发展，深度学习技术逐渐暴露了一定的安全隐患。现有的深度学习算法无法有效描述数据本质特征，导致算法面对恶意输入时可能无法给出正确结果。以当前深度学习面临的安全威胁为出发点，介绍了深度学习中的对抗样本问题，梳理了现有的对抗样本存在性解释，回顾了经典的对抗样本构造方法并对其进行了分类，简述了近年来部分对抗样本在不同场景中的应用实例，对比了若干对抗样本防御技术，最后归纳对抗样本研究领域存在的问题并对这一领域的发展趋势进行了展望。

Abstract

With the further promotion of deep learning technology in the fields of computer vision

network security and natural language processing

which has gradually exposed certain security risks.Existing deep learning algorithms can not effectively describe the essential characteristics of data or its inherent causal relationship.When the algorithm faces malicious input

it often fails to give correct judgment results.Based on the current security threats of deep learning

the adversarial example problem and its characteristics in deep learning applications were introduced

hypotheses on the existence of adversarial examples were summarized

classic adversarial example construction methods were reviewed and recent research status in different scenarios were summarized

several defense techniques in different processes were compared

and finally the development trend of adversarial example research were forecasted.

关键词

Keywords

references

SZEGEDY C , VANHOUCKE V , IOFFE S , et al . Rethinking the inception architecture for computer vision [C ] // The IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 2818 - 2826 .

TANG T A , MHAMDI L , MCLERNON D , et al . Deep learning approach for network intrusion detection in software defined networking [C ] // 2016 International Conference on Wireless Networks and Mobile Communications (WINCOM) . 2016 : 258 - 263 .

COLLOBERT R , WESTON J . A unified architecture for natural language processing:deep neural networks with multitask learning [C ] // The 25th International Conference on Machine Learning . 2008 : 160 - 167 .

CHEN C , SEFF A , KORNHAUSER A , et al . Deepdriving:learning affordance for direct perception in autonomous driving [C ] // The IEEE International Conference on Computer Vision . 2015 : 2722 - 2730 .

CHING T , HIMMELSTEIN D S , BEAULIEU-JONES B K , et al . Opportunities and obstacles for deep learning in biology and medicine [J ] . Journal of The Royal Society Interface , 2018 , 15 ( 141 ).

SZEGEDY C , ZAREMBA W , SUTSKEVER I , et al . Intriguing properties of neural networks [J ] . arXiv preprint arXiv:1312.6199 , 2013

KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial examples in the physical world [J ] . arXiv preprint arXiv:1607.02533 , 2016

ALZANTOT M , SHARMA Y , ELGOHARY A , et al . Generating natural language adversarial examples [J ] . arXiv preprint arXiv:1804.07998 , 2018

QIN Y , CARLINI N , GOODFELLOW I , et al . Imperceptible,robust,and targeted adversarial examples for automatic speech recognition [J ] . arXiv preprint arXiv:1903.10346 , 2019

LECUN Y , BENGIO Y , HINTON G . Deep learning [J ] . Nature , 2015 , 521 ( 7553 ):436.

PAPERNOT N , MCDANIEL P , GOODFELLOW I . Transferability in machine learning:from phenomena to black-box attacks using adversarial samples [J ] . arXiv preprint arXiv:1605.07277 , 2016

PAPERNOT N , MCDANIEL P , JHA S , et al . The limitations of deep learning in adversarial settings [C ] // The 1st IEEE European Symposium on Security and Privacy . 2016 .

宋蕾 , 马春光 , 段广晗 . 机器学习安全及隐私保护研究进展 [J ] . 网络与信息安全学报 , 2018 , 4 ( 8 ): 1 - 11 .

SONG L , MA C G , DUAN G H . Machine learning security and privacy:a survey [J ] . Chinese Journal of Network and Information Security , 2018 , 4 ( 8 ): 1 - 11 .

GU S , RIGAZIO L . Towards deep neural network architectures robust to adversarial examples [J ] . arXiv preprint arXiv:1412.5068 , 2014

GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [C ] // 2015 International Conference on Learning Representations . 2015 : 1 - 10 .

TABACOF P , VALLE E . Exploring the space of adversarial images [J ] . arXiv preprint arXiv:1510.05328 , 2015

TRAM`ER F , PAPERNOT N , GOODFELLOW I , et al . The space of transferable adversarial examples [J ] . arXiv preprint arXiv:1704.03453 , 2017

KROTOV D , HOPFIELD J J . Dense associative memory is robust to adversarial inputs [J ] . arXiv preprint arXiv:1701.00939 , 2017

LUO Y , BOIX X , ROIG G , et al . Foveation-based mechanisms alleviate adversarial examples [J ] . arXiv preprint arXiv:1511.06292 , 2015

TANAY T , GRIFFIN L . A boundary tilting perspective on the phenomenon of adversarial examples [J ] . arXiv preprint arXiv:1608.07690 , 2016

MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O , et al . Universal adversarial perturbations [C ] // The IEEE Conference on Computer Vision and Pattern Recognition . 2017 : 1765 - 1773 .

MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O , et al . Analysis of universal adversarial perturbations [J ] . arXiv preprint arXiv:1705.09554 , 2017

TRAM`ER F , KURAKIN A , PAPERNOT N , et al . Ensemble adversarial training:attacks and defenses [J ] . arXiv preprint arXiv:1705.07204 , 2017

MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O , et al . Robustness of classifiers to universal perturbations:a geometric perspective [C ] // International Conference on Learning Representations . 2018 .

SONG Y , KIM T , NOWOZIN S , et al . Pixeldefend:leveraging generative models to understand and defend against adversarial examples [J ] . arXiv preprint arXiv:1710.10766 , 2017

MENG D , CHEN H . Magnet:a two-pronged defense against adversarial examples [C ] // The 2017 ACM SIGSAC Conference on Computer and Communications Security . 2017 : 135 - 147 .

GHOSH P , LOSALKA A , BLACK M J . Resisting adversarial attacks using gaussian mixture variational autoencoders [J ] . arXiv preprint arXiv:1806.00081 , 2018

LEE H , HAN S , LEE J . Generative adversarial trainer:defense to adversarial perturbations with gan [J ] . arXiv preprint arXiv:1705.03387 , 2017

GILMER J , METZ L , FAGHRI F , et al . Adversarial spheres [J ] . arXiv preprint arXiv:1801.02774 , 2018

GILMER J , METZ L , FAGHRI F , et al . The relationship between high-dimensional geometry and adversarial examples [J ] . arXiv:1801.02774v3 , 2018

EYKHOLT K , EVTIMOV I , FERNANDES E , et al . Robust physical-world attacks on deep learning visual classification [C ] // The IEEE Conference on Computer Vision and Pattern Recognition . 2018 : 1625 - 1634 .

MOOSAVI-DEZFOOLI S M , FAWZI A , FROSSARD P . Deepfool:a simple and accurate method to fool deep neural networks [C ] // The 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2016 .

PAPERNOT N , MCDANIEL P , SWAMI A , et al . Crafting adversarial input sequences for recurrent neural networks [C ] // MILCOM 2016-2016 IEEE Military Communications Conference . 2016 : 49 - 54 .

GROSSE K , PAPERNOT N , MANOHARAN P , et al . Adversarial examples for malware detection [C ] // European Symposium on Research in Computer Security . 2017 : 62 - 79 .

RUSSAKOVSKY O , DENG J , SU H , et al . ImageNet large scale visual recognition challenge [J ] . International Journal of Computer Vision , 2015 , 115 ( 3 ): 211 - 252 .

PAPERNOT N , MCDANIEL P , JHA S , et al . The limitations of deep learning in adversarial settings [C ] // 2016 IEEE European Symposium on Security and Privacy . 2016 : 372 - 387 .

PAPERNOT N , MCDANIEL P , GOODFELLOW I , et al . Practical black-box attacks against machine learning [C ] // The 2017 ACM on Asia Conference on Computer and Communications Security . 2017 : 506 - 519 .

ILYAS A , ENGSTROM L , ATHALYE A , et al . Black-box adversarial attacks with limited queries and information [J ] . arXiv preprint arXiv:1804.08598 , 2018

BALUJA S , FISCHER I . Adversarial transformation networks:Learning to generate adversarial examples [J ] . arXiv preprint arXiv:1703.09387 , 2017

XIAO C , LI B , ZHU J Y , et al . Generating adversarial examples with adversarial networks [C ] // The 27th International Joint on Artificial Intelligence Main track . 2019 : 3805 - 3911 .

ZHAO P , FU Z , HU Q , et al . Detecting adversarial examples via key-based network [J ] . arXiv preprint arXiv:1806.00580 , 2018

MENG D , CHEN H . Magnet:a two-pronged defense against adversarial examples [C ] // The 2017 ACM SIGSAC Conference on Computer and Communications Security . 2017 : 135 - 147 .

XU W , EVANS D , QI Y . Feature squeezing:detecting adversarial examples in deep neural networks [J ] . arXiv preprint arXiv:1704.01155 , 2017

HOSSEIN H , CHEN Y , KANNAN S , et al . Blocking transferability of adversarial examples in black-box learning systems [J ] . arXiv:1703.04318 , 2017

SABOUR S , NICHOLAS F , HINTON G E . Dynamic routing between capsules [C ] // Neural Information Processing Systems . 2017 .

NICHOLAS F , SABOUR S , HINTON G . DARCCC:detecting adversaries by reconstruction from class conditional capsules [J ] . arXiv preprint arXiv:1811.06969 , 2018

TRAMÈR F , KURAKIN A , PAPERNOT N , et al . Ensemble adversarial raining:attacks and defenses [J ] . arXiv:1705.07204 , 2017

SINHA A , CHEN Z , BADRINARAYANAN V , et al . Gradient adversarial training of neural networks [J ] . arXiv preprint arXiv:1806.08028 , 2018

KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial machine learning at scale [J ] . arXiv preprint arXiv:1611.01236 , 2016

PAPERNOT N , MCDANIEL P , WU X , et al . Distillation as a defense to adversarial perturbations against deep neural networks [C ] // 2016 IEEE Symposium on Security and Privacy . 2016 : 582 - 597 .

HINTON G E , VINYALS O , DEAN J . Distilling the knowledge in a neural network [J ] . arXiv:1503.02531 ,

LEE H , HAN S , LEE J . Generative adversarial trainer:defense to adversarial perturbations with GAN [J ] . arXiv preprint arXiv:1705.03387 , 2017

Views

1886

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Survey of optical-based physical domain adversarial attacks and defense

Adversarial example defense algorithm for MNIST based on image reconstruction

Phishing website detection based on two-way feature extraction and adaptive fusion

Anomaly detection in DNS: an empirical study

FL-IoV: comprehensive intrusion detection method for internet of vehicles based on federated learning

Related Author

CHEN Jinyin

ZHAO Xiaoming

ZHENG Haibin

GUO Haifeng

Zhongyuan QIN

Zhaoxiang HE

Tao LI

Liquan CHEN

Related Institution

Institute of Cyberspace Security, Zhejiang University of Technology

College of Information Engineering, Zhejiang University of Technology

School of Cyber Science and Engineering, Southeast University

Network Communication and Security Purple Mountain Laboratory

School of Information Engineering, Ningxia University

⁰