基于边界值不变量的对抗样本检测方法

严飞; 张铭伦; 张立强

doi:10.11959/j.issn.2096-109x.2020012

您当前的位置：

首页 >

文章列表页 >

基于边界值不变量的对抗样本检测方法

学术论文 | 更新时间：2024-06-03

- 基于边界值不变量的对抗样本检测方法
- Adversarial examples detection method based on boundary values invariants
- 网络与信息安全学报 2020年6卷第1期页码：38-45
- 作者机构：
- 作者简介：
  
  [ "严飞（1980- ），男，湖北武汉人，武汉大学副教授、硕士生导师，主要研究方向为系统安全、可信计算、系统安全验证与形式化分析和移动目标防御" ]
  [ "张铭伦（1995- ），男，江苏连云港人，武汉大学硕士生，主要研究方向为人工智能系统本身的安全防护问题" ]
  [ "张立强（1979- ），男，黑龙江哈尔滨人，武汉大学讲师，主要研究方向为系统安全、可信计算和安全测评" ]
- 基金信息：
  
  国家重点基础研究发展计划（“973”计划）基金资助项目;The Major State Basic Research Development Program of China (973 Program)(2014CB340601);国家自然科学基金资助项目;The National Natural Science Foundation of China(61272452)
- DOI：10.11959/j.issn.2096-109x.2020012
  中图分类号： TP309.2
- 网络首发：2020-02，
  
  纸质出版：2020-02-15
- 稿件说明：
移动端阅览
严飞, 张铭伦, 张立强. 基于边界值不变量的对抗样本检测方法[J]. 网络与信息安全学报, 2020,6(1):38-45.

Fei YAN, Minglun ZHANG, Liqiang ZHANG. Adversarial examples detection method based on boundary values invariants[J]. Chinese Journal of Network and Information Security, 2020, 6(1): 38-45.
严飞, 张铭伦, 张立强. 基于边界值不变量的对抗样本检测方法[J]. 网络与信息安全学报, 2020,6(1):38-45. DOI： 10.11959/j.issn.2096-109x.2020012.

Fei YAN, Minglun ZHANG, Liqiang ZHANG. Adversarial examples detection method based on boundary values invariants[J]. Chinese Journal of Network and Information Security, 2020, 6(1): 38-45. DOI： 10.11959/j.issn.2096-109x.2020012.

摘要

目前，深度学习成为计算机领域研究与应用最广泛的技术之一，在图像识别、语音、自动驾驶、文本翻译等方面都取得良好的应用成果。但人们逐渐发现深度神经网络容易受到微小扰动图片的影响，导致分类出现错误，这类攻击手段被称为对抗样本。对抗样本的出现可能会给安全敏感的应用领域带来灾难性的后果。现有的防御手段大多需要对抗样本本身作为训练集，这种对抗样本相关的防御手段是无法应对未知对抗样本攻击的。借鉴传统软件安全中的边界检查思想，提出了一种基于边界值不变量的对抗样本检测防御方法，该方法通过拟合分布来寻找深度神经网络中的不变量，且训练集的选取与对抗样本无关。实验结果表明，在 LeNet、vgg19 模型和 Mnist、Cifar10 数据集上，与其他对抗检测方法相比，提出的方法可有效检测目前的常见对抗样本攻击，并且具有低误报率。

Abstract

Nowadays

deep learning has become one of the most widely studied and applied technologies in the computer field.Deep neural networks(DNNs) have achieved greatly noticeable success in many applications such as image recognition

speech

self-driving and text translation.However

deep neural networks are vulnerable to adversarial examples that are generated by perturbing correctly classified inputs to cause DNN modes to misbehave.A boundary check method based on traditional programs by fitting the distribution to find the invariants in the deep neural network was proposed and it use the invariants to detect adversarial examples.The selection of training sets was irrelevant to adversarial examples.The experiment results show that proposed method can effectively detect the current adversarial example attacks on LeNet

vgg19 model

Mnist

Cifar10 dataset

and has a low false positive rate.

关键词

Keywords

references

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . Imagenet classification with deep convolutional neural networks [C ] // Advances in Neural Information Processing Systems . 2012 : 1097 - 1105 .

BOJARSKI M , Del TESTA D , DWORAKOWSKI D , et al . End to end learning for self-driving cars [J ] . arXiv preprint arXiv:1604.07316 , 2016 .

DAHL G E , STOKES J W , DENG L , et al . Large-scale malware classification using random projections and neural networks [C ] // 2013 IEEE International Conference on Acoustics,Speech and Signal Processing . 2013 : 3422 - 3426 .

MIRSKY Y , DOITSHMAN T , ELOVICI Y , et al . Kitsune:an ensemble of autoencoders for online network intrusion detection [J ] . arXiv preprint arXiv:1802.09089 , 2018 .

SZEGEDY C , ZAREMBA W , SUTSKEVER I , et al . Intriguing properties of neural networks [J ] . arXiv preprint arXiv:1312.6199 , 2013 .

DHILLON G S , AZIZZADENESHELI K , LIPTON Z C , et al . Stochastic activation pruning for robust adversarial defense [J ] . arXiv preprint arXiv:1803.01442 , 2018 .

GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [J ] . arXiv:preprint arXiv:1412.6572 , 2014 .

KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial examples in the physical world [J ] . arXiv preprint arXiv:1607.02533 , 2016 .

CARLINI N , WAGNER D . Defensive distillation is not robust to adversarial examples [J ] . arXiv preprint arXiv:1607.04311 , 2016 .

PAPERNOT N , MCDANIEL P , JHA S , et al . The limitations of deep learning in adversarial settings [C ] // 2016 IEEE European Symposium on Security and Privacy (EuroS＆P) . 2016 : 372 - 387 .

LIU Y , MA S , AAFER Y , et al . Trojaning attack on neural networks [C ] // Network and Distributed System Security Symposium . 2018 .

GU S , RIGAZIO L . Towards deep neural network architectures robust to adversarial examples [J ] . arXiv preprint arXiv:1412.5068 , 2014 .

FEINMAN R , CURTIN R R , SHINTRE S , et al . Detecting adversarial samples from artifacts [J ] . arXiv preprint arXiv:1703.00410 , 2017 .

GROSSE K , MANOHARAN P , PAPERNOT N , et al . On the (statistical) detection of adversarial examples [J ] . arXiv preprint arXiv:1702.06280 , 2016 .

MA X , LI B , WANG Y , et al . Characterizing adversarial subspaces using local intrinsic dimensionality [J ] . arXiv preprint arXiv:1801.02613 , 2018 .

XU W , EVANS D , QI Y . Feature squeezing:detecting adversarial examples in deep neural networks [J ] . arXiv preprint arXiv:1704.01155 , 2017 .

MENG D , CHEN H . Magnet:a two-pronged defense against adversarial examples [C ] // The 2017 ACM SIGSAC Conference on Computer and Communications Security . 2017 : 135 - 147 .

LIAO F , LIANG M , DONG Y , et al . Defense against adversarial attacks using high-level representation guided denoiser [C ] // The IEEE Conference on Computer Vision and Pattern Recognition . 2018 : 1778 - 1787 .

LECUN Y , BOTTOU L , BENGIO Y , et al . Gradient-based learning applied to document recognition [J ] . Proceedings of the IEEE , 1998 , 86 ( 11 ): 2278 - 2324 .

BROWN T B , MANÉ D , ROY A , et al . Adversarial patch [J ] . arXiv preprint arXiv:1712.09665 , 2017 .

EYKHOLT K , EVTIMOV I , FERNANDES E , et al . Robust physical-world attacks on deep learning models [J ] . arXiv preprint arXiv:1707.08945 , 2017 .

PEI K , CAO Y , YANG J , et al . Deepxplore:automated whitebox testing of deep learning systems [C ] // The 26th Symposium on Operating Systems Principles . 2017 : 1 - 18 .

BIGGIO B , ROLI F . Wild patterns:ten years after the rise of adversarial machine learning [J ] . Pattern Recognition , 2018 , 84 : 317 - 331 .

MOOSAVI-DEZFOOLI S M , FAWZI A , FROSSARD P . DeepFool:a simple and accurate method to fool deep neural networks [C ] // IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 2574 - 2582 .

CARLINI N , WAGNER D . Towards evaluating the robustness of neural networks [C ] // 2017 IEEE Symposium on Security and Privacy (SP) . 2017 : 39 - 57 .

KINGMA D P , BA J . Adam:a method for stochastic optimization [J ] . arXiv preprint arXiv:1412.6980 , 2014 .

ROUHANI B D , SAMRAGH M , JAVAHERIPIM , et al . Deepfense:online accelerated defense against adversarial deep learning [C ] // IEEE/ACM International Conference on Computer-Aided Design (ICCAD) . 2018 : 1 - 8 .

SONG Y , KIM T , NOWOZIN S , et al . Pixeldefend:leveraging generative models to understand and defend against adversarial examples [J ] . arXiv preprint arXiv:1710.10766 , 2017 .

XIE C , WANG J , ZHANG Z , et al . Mitigating adversarial effects through randomization [J ] . arXiv preprint arXiv:1711.01991 , 2017 .

PAPERNOT N , MCDANIEL P , SINHA A , et al . Towards the science of security and privacy in machine learning [J ] .,2016. arXiv preprint arXiv:1611.03814 , 2016 .

PAPERNOT N , MCDANIEL P , WU X , et al . Distillation as a defense to adversarial perturbations against deep neural networks [C ] // 2016 IEEE Symposium on Security and Privacy (SP) . 2016 : 582 - 597 .

PAPERNOT N , MCDANIEL P , GOODFELLOW I , et al . Practical black-box attacks against machine learning [C ] // ACM on Asia Conference on Computer and Communications Security . 2017 : 506 - 519 .

ATHALYE A , CARLINI N , WAGNER D . Obfuscated gradients give a false sense of security:circumventing defenses to adversarial examples [J ] . arXiv preprint arXiv:1802.00420 , 2018 .

BHAGOJI A N , CULLINA D , MITTAL P . Dimensionality reduction as a defense against evasion attacks on machine learning classifiers [J ] . arXiv preprint arXiv:1704.02654 , 2017 .

GONG Z , WANG W , KU W S . Adversarial and clean data are not twins [J ] . arXiv preprint arXiv:1704.04960 , 2017 .

HENDRYCKS D , GIMPEL K . Early methods for detecting adversarial images [J ] . arXiv preprint arXiv:1608.00530 , 2016 .

TAX D M J , DUIN R P W . Support vector domain description [J ] . Pattern Recognition Letters , 1999 , 20 ( 11-13 ): 1191 - 1199 .

CARLINI N , WAGNER D . Adversarial examples are not easily detected:bypassing ten detection methods [C ] // The 10th ACM Workshop on Artificial Intelligence and Security . 2017 : 3 - 14 .

LU P H , CHEN P Y , YU C M . On the limitation of local intrinsic dimensionality for characterizing the subspaces of adversarial examples [J ] . arXiv preprint arXiv:1803.09638 , 2018 .

GUO C , RANA M , CISSE M , et al . Countering adversarial images using input transformations [J ] . arXiv preprint arXiv:1711.00117 , 2017 .

TAO G , MA S , LIU Y , et al . Attacks meet interpretability:attribute-steered detection of adversarial samples [C ] // Advances in Neural Information Processing Systems . 2018 : 7717 - 7728 .

GILMER J , METZ L , FAGHRI F , et al . Adversarial spheres [J ] . arXiv preprint arXiv:1801.02774 , 2018 .

PERERA P , PATEL V M . Learning deep features for one-class classification [J ] . IEEE Transactions on Image Processing , 2019 , 28 ( 11 ): 5450 - 5463 .

TAX D M J , DUIN R P W . Data domain description using support vectors [C ] // ESANN . 1999 ， 99 : 251 - 256 .

KRIZHEVSKY A , HINTON G . Learning multiple layers of features from tiny images [R ] . Technical Report,University of Toronto , 2009 .

RAUBER J , BRENDEL W , BETHGE M . Foolbox:a Python toolbox to benchmark the robustness of machine learning models [J ] . arXiv preprint arXiv:1707.04131 , 2017 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [J ] . arXiv preprint arXiv:1409.1556 , 2014 .

浏览量

776

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据