KRIZHEVSKY A , SUTSKEVER I , HINTON G E . Imagenet classification with deep convolutional neural networks [C ] // Advances in Neural Information Processing Systems . 2012 : 1097 - 1105 .

DAHL G E , YU D , DENG L , et al . Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition [J ] . IEEE Transactions on Audio,Speech,and Language Processing , 2011 , 20 : 30 - 42 .

HINTON G , DENG L , YU D , et al . Deep neural networks for acoustic modeling in speech recognition:The shared views of four research groups [J ] . IEEE Signal Processing Magazine , 2012 , 29 : 82 - 97 .

COLLOBERT R , WESTON J . A unified architecture for natural language processing:deep neural networks with multitask learning [C ] // Proceedings of the 25th International Conference on Machine Learning . 2008 : 160 - 167 .

DAHL G E , STOKES J W , DENG L , et al . Large-scale malware classification using random projections and neural networks [C ] // 2013 IEEE International Conference on Acoustics,Speech and Signal Processing . 2013 : 3422 - 3426 .

SILVER D , HUANG A , MADDISON C J , et al . Mastering the game of go with deep neural networks and tree search [J ] . Nature , 2016 ， 529 :484.

VINYALS O , BABUSCHKIN I , CZARNECKI W M , et al . Grandmaster level in StarCraft II using multi-agent reinforcement learning [J ] . Nature , 2019 , 575 : 350 - 354 .

DENG J , DONG W , SOCHER R , et al . Imagenet:a large-scale hierarchical image database [C ] // 2009 IEEE Conference on Computer Vision and Pattern Recognition . 2009 : 248 - 255 .

LECUN Y , BOSER B , DENKER J S , et al . Backpropagation applied to handwritten zip code recognition [J ] . Neural Computation , 1989 , 1 : 541 - 551 .

SZEGEDY C , ZAREMBA W , SUTSKEVER I , et al . Intriguing properties of neural networks [J ] . arXiv preprint arXiv:1312.6199 , 2013

GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [J ] . arXiv preprint arXiv:1412.6572 , 2014

MOOSAVI-DEZFOOLI S-M , FAWZI A , FAWZI O , et al . Universal adversarial perturbations [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2017 : 1765 - 1773 .

SU J , VARGAS D V , SAKURAI K . One pixel attack for fooling deep neural networks [J ] . IEEE Transactions on Evolutionary Computation , 2019 , 23 : 828 - 841 .

CARLINI N , WAGNER D . Adversarial examples are not easily detected:Bypassing ten detection methods [C ] // Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security . 2017 : 3 - 14 .

ATHALYE A , CARLINI N . On the robustness of the CVPR 2018 white-box adversarial example defenses [J ] . arXiv preprint arXiv:1804.03286 , 2018

ATHALYE A , CARLINI N , WAGNER D . Obfuscated gradients give a false sense of security:Circumventing defenses to adversarial examples [J ] . arXiv preprint arXiv:1802.00420 , 2018

FLETCHER R . Practical methods of optimization [M ] . John Wiley＆ Sons , 2013 .

KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial machine learning at scale [J ] . arXiv preprint arXiv:1611.01236 , 2016

PAPERNOT N , MCDANIEL P , JHA S , et al . The limitations of deep learning in adversarial settings [C ] // 2016 IEEE European Symposium on Security and Privacy (EuroS＆P) . 2016 : 372 - 387 .

CARLINI N , WAGNER D . Towards evaluating the robustness of neural networks [C ] // 2017 IEEE Symposium on Security and Privacy (SP) . 2017 : 39 - 57 .

MOOSAVI-DEZFOOLI S-M , FAWZI A , FROSSARD P . Deepfool:a simple and accurate method to fool deep neural networks [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 2574 - 2582 .

ATHALYE A , ENGSTROM L , ILYAS A , et al . Synthesizing robust adversarial examples [J ] . arXiv preprint arXiv:1707.07397 , 2017

MCLAREN K . XIII—The development of the CIE 1976 (L* a* b*) uniform colour space and colour-difference formula [J ] . Journal of the Society of Dyers and Colourists , 1976 , 92 : 338 - 341 .

CHEN P-Y , ZHANG H , SHARMA Y , et al . Zoo:zeroth order optimization based black-box attacks to deep neural networks without training substitute models [C ] // Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security . 2017 : 15 - 26 .

KINGMA D P , BA J . Adam:A method for stochastic optimization [J ] . arXiv preprint arXiv:1412.6980 , 2014

BRENDEL W , RAUBER J , BETHGE M . Decision-based adversarial attacks:reliable attacks against black-box machine learning models [J ] . arXiv preprint arXiv:1712.04248 , 2017

BRUNNER T , DIEHL F , LE M T , et al . Guessing smart:biased sampling for efficient black-box adversarial attacks [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2019 : 4958 - 4966 .

PERLIN K . An image synthesizer [J ] . ACM Siggraph Computer Graphics , 1985 , 19 : 287 - 296 .

CHEN J , JORDAN M I , WAINWRIGHT M J . HopSkipJumpAttack:a query-efficient decision-based attack [J ] . arXiv preprint arXiv:1904.02144,3:2019 ,

KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial examples in the physical world [J ] . arXiv preprint arXiv:1607.02533 , 2016

SZEGEDY C , VANHOUCKE V , IOFFE S , et al . Rethinking the inception architecture for computer vision [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 2818 - 2826 .

PAPERNOT N , MCDANIEL P , GOODFELLOW I , et al . Practical black-box attacks against machine learning [C ] // Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security . 2017 : 506 - 519 .

LIU Y , CHEN X , LIU C , et al . Delving into transferable adversarial examples and black-box attacks [J ] . arXiv preprint arXiv:1611.02770 , 2016

LI X , JI S , HAN M , et al . Adversarial examples versus cloud-based detectors:a black-box empirical study [J ] . IEEE Transactions on Dependable and Secure Computing , 2019

WEI X , LIANG S , CHEN N , et al . Transferable adversarial attacks for image and video object detection [J ] . arXiv preprint arXiv:1811.12641 , 2018

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial nets [C ] // Advances in Neural Information Processing Systems . 2014 : 2672 - 2680 .

REN S , HE K , GIRSHICK R , et al . Faster r-CNN:Towards real-time object detection with region proposal networks [C ] // Advances in Neural Information Processing Systems . 2015 : 91 - 99 .

LIU W , ANGUELOV D , ERHAN D , et al . SSD:single shot multibox detector [C ] // European Conference on Computer Vision . 2016 : 21 - 37 .

THYS S , VAN RANST W , GOEDEMÉ T . Fooling automated surveillance cameras:adversarial patches to attack person detection [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops . 2019 .

REDMON J , FARHADI A . YOLO9000:better,faster,stronger [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2017 : 7263 - 7271 .

EVTIMOV I , EYKHOLT K , FERNANDES E , et al . Robust physical-world attacks on deep learning models [J ] . arXiv preprint arXiv:1707.08945 , 2017

LU J , SIBAI H , FABRY E , et al . Standard detectors aren’t (currently) fooled by physical adversarial stop signs [J ] . arXiv preprint arXiv:1710.03337 , 2017

EYKHOLT K , EVTIMOV I , FERNANDES E , et al . Note on attacking object detectors with adversarial stickers [J ] . arXiv preprint arXiv:1712.08062 , 2017

CHEN S T , CORNELIUS C , MARTIN J , et al . Shapeshifter:robust physical adversarial attack on faster r-CNN object detector [C ] // Joint European Conference on Machine Learning and Knowledge Discovery in Databases . 2018 : 52 - 68 .

BROWN T B,MANÉ D , ROY A , et al . Adversarial Patch [J ] . CoRR,abs/1712.09665:2017 .

HENDRIK METZEN J , CHAITHANYA KUMAR M , BROX T , et al . Universal adversarial perturbations against semantic image segmentation [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2017 : 2755 - 2764 .

XIE C , WANG J , ZHANG Z , et al . Adversarial examples for semantic segmentation and object detection [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2017 : 1369 - 1378 .

CISSE M , ADI Y , NEVEROVA N , et al . Houdini:Fooling deep structured prediction models [J ] . arXiv preprint arXiv:1707.05373 , 2017

SHARIF M , BHAGAVATULA S , BAUER L , et al . Accessorize to a crime:Real and stealthy attacks on state-of-the-art face recognition [C ] // Proceedings of the 2016 ACM Sigsac Conference on Computer and Communications Security . 2016 : 1528 - 1540 .

ZHOU Z , TANG D , WANG X , et al . Invisible mask:Practical attacks on face recognition with infrared [J ] . arXiv preprint arXiv:1803.04683 , 2018

WANG Q , GUO W , ZHANG K , et al . Adversary resistant deep neural networks with an application to malware detection [C ] // Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . 2017 : 1145 - 1153 .

SRIVASTAVA N , HINTON G , KRIZHEVSKY A , et al . Dropout:a simple way to prevent neural networks from overfitting [J ] . The Journal of Machine Learning Research , 2014 , 15 : 1929 - 1958 .

PRAKASH A , MORAN N , GARBER S , et al . Deflecting adversarial attacks with pixel deflection [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 : 8571 - 8580 .

HO J , KANG D-K . Pixel Redrawn For A Robust Adversarial Defense [J ] .

DZIUGAITE G K , GHAHRAMANI Z , ROY D M . A study of the effect of jpg compression on adversarial images [J ] . arXiv preprint arXiv:1608.00853 , 2016

DAS N , SHANBHOGUE M , CHEN S T , et al . Keeping the bad guys out:Protecting and vaccinating deep learning with jpeg compression [J ] . arXiv preprint arXiv:1705.02900 , 2017

GUO C , RANA M , CISSE M , et al . Countering adversarial images using input transformations [J ] . arXiv preprint arXiv:1711.00117 , 2017

RUDIN L I , OSHER S , FATEMI E . Nonlinear total variation based noise removal algorithms [J ] . Physica D:Nonlinear Phenomena , 1992 , 60 : 259 - 268 .

EFROS A A , FREEMAN W T . Image quilting for texture synthesis and transfer [C ] // Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques . 2001 : 341 - 346 .

XU W , EVANS D , QI Y . Feature squeezing:Detecting adversarial examples in deep neural networks [J ] . arXiv preprint arXiv:1704.01155 , 2017

BUADES A , COLL B , MOREL J M . A non-local algorithm for image denoising [C ] // 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) . 2005 : 60 - 65 .

RAFF E , SYLVESTER J , FORSYTH S , et al . Barrage of random transforms for adversarially robust defense [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2019 : 6528 - 6537 .

ANTONINI M , BARLAUD M , MATHIEU P , et al . Image coding using wavelet transform [J ] . IEEE Transactions on Image Processing , 1992 , 1 : 205 - 220 .

HUANG T , YANG G , TANG G . A fast two-dimensional median filtering algorithm [J ] . IEEE Transactions on Acoustics,Speech,and Signal Processing , 1979 , 27 : 13 - 18 .

HE W , WEI J , CHEN X , et al . Adversarial example defense:Ensembles of weak defenses are not strong [C ] // 11th $$USENIX$$ Workshop on Offensive Technologies ($$WOOT$$ 17) , 2017 .

AKHTAR N , LIU J , MIAN A . Defense against universal adversarial perturbations [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018 : 3389 - 3398 .

CORTES C , VAPNIK V . Support-vector networks [J ] . Machine Learning , 1995 , 20 : 273 - 297 .

VINCENT P , LAROCHELLE H , BENGIO Y , et al . Extracting and composing robust features with denoising autoencoders [C ] // Proceedings of the 25th International Conference on Machine Learning . 2008 : 1096 - 1103 .

LIAO F , LIANG M , DONG Y , et al . Defense against adversarial attacks using high-level representation guided denoiser [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 : 1778 - 1787 .

RONNEBERGER O , FISCHER P , BROX T . U-net:convolutional networks for biomedical image segmentation [C ] // International Conference on Medical Image Computing and Computer-assisted Intervention . 2015 : 234 - 241 .

SAMANGOUEI P , KABKAB M , CHELLAPPA R . Defense-gan:protecting classifiers against adversarial attacks using generative models [J ] . arXiv preprint arXiv:1805.06605 , 2018

KRIZHEVSKY A , NAIR V , HINTON G . CIFAR-10 (Canadian Institute for Advanced Research) [D ] . Toronto:University of Toronto , 2009 .

YANN L , BERNHARD B , JOHN S D , et al . Backpropagation applied to handwritten zip code recognition [J ] . Neural Computation , 1989 , 1 ( 4 ): 541 - 551 .

BAO R , LIANG S , WANG Q . Featurized Bidirectional GAN:adversarial defense via adversarially learned semantic inference [J ] . arXiv preprint arXiv:1805.07862 , 2018

DUMOULIN V , BELGHAZI I , POOLE B , et al . Adversarially learned inference [J ] . arXiv preprint arXiv:1606.00704 , 2016

DONAHUE J , KRÄHENBÜHL P , DARRELL T . Adversarial feature learning [J ] . arXiv preprint arXiv:1605.09782 , 2016

MUSTAFA A , KHAN S H , HAYAT M , et al . Image super-resolution as a defense against adversarial attacks [J ] . IEEE Transactions on Image Processing , 2019 , 29 : 1711 - 1724 .

LIM B , SON S , KIM H , et al . Enhanced deep residual networks for single image super-resolution [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops . 2017 : 136 - 144 .

CHANG S G , YU B , VETTERLI M . Adaptive wavelet thresholding for image denoising and compression [J ] . IEEE Transactions on Image Processing , 2000 , 9 : 1532 - 1546 .

IOFFE S , SZEGEDY C . Batch normalization:Accelerating deep network training by reducing internal covariate shift [J ] . arXiv preprint arXiv:1502.03167 , 2015

MADRY A , MAKELOV A , SCHMIDT L , et al . Towards deep learning models resistant to adversarial attacks [J ] . arXiv preprint arXiv:1706.06083 , 2017

CHANG T J , HE Y , LI P . Efficient Two-Step Adversarial Defense for Deep Neural Networks [J ] . arXiv preprint arXiv:1810.03739 , 2018

HUANG R , XU B , SCHUURMANS D , et al . Learning with a strong adversary [J ] . CoRR,abs/1511.03034:2015 .

SHAHAM U , YAMADA Y , NEGAHBAN S . Understanding adversarial training:Increasing local stability of supervised models through robust optimization [J ] . Neurocomputing , 2018 , 307 : 195 - 204 ,.

KANNAN H , KURAKIN A , GOODFELLOW I . Adversarial logit pairing [J ] . arXiv preprint arXiv:1803.06373 , 2018

TRAMÈR F , KURAKIN A , PAPERNOT N , et al . Ensemble adversarial training:Attacks and defenses [J ] . arXiv preprint arXiv:1705.07204 , 2017

LI P , YI J , ZHOU B , et al . Improving the Robustness of Deep Neural Networks via Adversarial Training with Triplet Loss [J ] . arXiv preprint arXiv:1905.11713 , 2019

SONG C , HE K , WANG L , et al . Improving the generalization of adversarial training with domain adaptation [J ] . arXiv preprint arXiv:1810.00740 , 2018

ROZSA A , GUNTHER M , Boult T E . Towards robust deep neural networks with BANG [J ] . arXiv preprint arXiv:1612.00138 , 2016

TOMAR V S , ROSE R C . Manifold regularized deep neural networks [C ] // Fifteenth Annual Conference of the International Speech Communication Association . 2014 .

NETZER Y , WANG T , COATES A , et al . Reading digits in natural images with unsupervised feature learning [R ] . 2011 .

SANKARANARAYANAN S , JAIN A , CHELLAPPA R , et al . Regularizing deep networks using efficient layerwise adversarial training [C ] // Thirty-Second AAAI Conference on Artificial Intelligence . 2018 :

LIU C , JAJA J . Feature prioritization and regularization improve standard accuracy and adversarial robustness [J ] . arXiv preprint arXiv:1810.02424 , 2018

PAPERNOT N , MCDANIEL P , WU X , et al . Distillation as a defense to adversarial perturbations against deep neural networks [C ] // 2016 IEEE Symposium on Security and Privacy (SP) . 2016 : 582 - 597 .

HINTON G , VINYALS O , DEAN J . Distilling the knowledge in a neural network [J ] . arXiv preprint arXiv:1503.02531 , 2015

CARLINI N , WAGNER D . Defensive distillation is not robust to adversarial examples [J ] . arXiv preprint arXiv:1607.04311 , 2016

XIE C , WU Y , MAATEN L V D , et al . Feature denoising for improving adversarial robustness [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2019 : 501 - 509 .

GAO J , WANG B , LIN Z , et al . Masking deep neural network models for robustness against adversarial samples [J ] . arXiv preprint arXiv:1702.06763 ,

SUN B , TSAI N-H , LIU F , et al . Adversarial Defense by Stratified Convolutional Sparse Coding [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2019 ： 11447 - 11456 .

CHOUDHURY B , SWANSON R , HEIDE F , et al . Consensus convolutional sparse coding [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2017 : 4280 - 4288 .

GU S , RIGAZIO L . Towards deep neural network architectures robust to adversarial examples [J ] . arXiv preprint arXiv:1412.5068 , 2014

RIFAI S , VINCENT P , MULLER X , et al . Contractive auto-encoders:explicit invariance during feature extraction [C ] // Proceedings of the 28th International Conference on International Conference on Machine Learning . 2011 : 833 - 840 .

HOSSEINI H , CHEN Y , KANNAN S , et al . Blocking transferability of adversarial examples in black-box learning systems [J ] . arXiv preprint arXiv:1703.04318 , 2017

TIAN S , YANG G , CAI Y . Detecting adversarial examples through image transformation [C ] // Thirty-Second AAAI Conference on Artificial Intelligence , 2018 .

CHEN S , CARLINI N , WAGNER D . Stateful detection of black-box adversarial attacks [J ] . arXiv preprint arXiv:1907.05587 , 2019

ILYAS A , ENGSTROM L , ATHALYE A , et al . Black-box adversarial attacks with limited queries and information [J ] . arXiv preprint arXiv:1804.08598 , 2018

METZEN J H , GENEWEIN T , FISCHER V , et al . On detecting adversarial perturbations [J ] . arXiv preprint arXiv:1702.04267 , 2017

LI X , LI F . Adversarial examples detection in deep networks with convolutional filter statistics [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2017 : 5764 - 5772 .

NGUYEN A , YOSINSKI J , CLUNE J . Deep neural networks are easily fooled:High confidence predictions for unrecognizable images [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2015 : 427 - 436 .

FEINMAN R , CURTIN R R , SHINTRE S , et al . Detecting adversarial samples from artifacts [J ] . arXiv preprint arXiv:1703.00410 , 2017

ZHENG Z , HONG P . Robust detection of adversarial attacks by modeling the intrinsic properties of deep neural networks [C ] // Advances in Neural Information Processing Systems , 2018 : 7913 - 7922 .

MENG D , CHEN H . Magnet:a two-pronged defense against adversarial examples [C ] // Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security . 2017 : 135 - 147 .

CARLINI N , WAGNER D . Magnet and" efficient defenses against adversarial attacks" are not robust to adversarial examples [J ] . arXiv preprint arXiv:1711.08478 , 2017

HUANG B , WANG Y , WANG W . Model-agnostic adversarial detection by random perturbations [C ] // Proceedings of the 28th International Joint Conference on Artificial Intelligence , 2019 : 4689 - 4696 .

LIANG B , LI H , SU M , et al . Detecting adversarial image examples in deep networks with adaptive noise reduction [J ] . arXiv preprint arXiv:1705.08378 , 2017

CHENG M , LE T , CHEN P Y , et al . Query-efficient hard-label black-box attack:An optimization-based approach [J ] . arXiv preprint arXiv:1807.04457 , 2018