Novel defense based on softmax activation transformation

Jinyin CHEN; Changan WU; Haibin ZHENG

doi:10.11959/j.issn.2096-109x.2022016

您当前的位置：

首页 >

文章列表页 >

Novel defense based on softmax activation transformation

Topic: Cybersecurity——Attack and Defense Technologies | 更新时间：2024-06-03

- Novel defense based on softmax activation transformation
- Chinese Journal of Network and Information Security Vol. 8, Issue 2, Pages: 48-63(2022)
- 作者机构：
  
  1. 浙江工业大学网络空间安全研究院，浙江杭州 310023
  2. 浙江工业大学信息工程学院，浙江杭州310023
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2022016
  CLC： TP181
- Online First：2022-04，
  
  Published：15 April 2022
- 稿件说明：
移动端阅览
Jinyin CHEN, Changan WU, Haibin ZHENG. Novel defense based on softmax activation transformation[J]. Chinese Journal of Network and Information Security, 2022, 8(2): 48-63.
DOI：

Jinyin CHEN, Changan WU, Haibin ZHENG. Novel defense based on softmax activation transformation[J]. Chinese Journal of Network and Information Security, 2022, 8(2): 48-63. DOI： 10.11959/j.issn.2096-109x.2022016.

摘要

深度学习广泛应用于图像处理、自然语言处理、网络挖掘等各个领域并取得良好效果，但其容易受到对抗攻击、存在安全漏洞的问题引起广泛关注。目前已有一些有效的防御方法，包括对抗训练、数据变化、模型增强等方法。但是，依然存在一些问题，如提前已知攻击方法与对抗样本才能实现有效防御、面向黑盒攻击的防御能力差、以牺牲部分正常样本的处理性能为代价、防御性能无法验证等。因此，提出可验证的、对抗样本不依赖的防御方法是关键。提出了 softmax 激活变换防御（SAT，softmax activation transformation），这是一种针对黑盒攻击的轻量级的快速防御。SAT不参与模型的训练，在推理阶段对目标模型的输出概率进行隐私保护加固并重新激活，通过softmax激活变换与深度模型防御的连接定义，证明通过softmax函数的变换后能实现概率信息的隐私保护从而防御黑盒攻击。SAT的实现不依赖对抗攻击方法与对抗样本，不仅避免了制作大量对抗样本的负担，也实现了攻击的事前防御。通过理论证明 SAT 的激活具有单调性，从而保证其防御过程中正常样本的识别准确率。在激活过程中，提出可变的softmax激活函数变换系数保护策略，在给定范围内随机选择隐私保护变换系数实现动态防御。最重要的一点，SAT 是一种可验证的防御，能够基于概率信息隐私保护和softmax激活变换推导其防御的有效性和可靠性。为了评估SAT的有效性，在MNIST、CIFAR10和ImageNet数据集上进行了针对9种黑盒攻击的防御实验，令所有攻击方法的平均攻击成功率从 87.06%降低为 5.94%，与多种先进黑盒攻击防御方法比较，验证了所提方法可以达到最优防御性能。

Abstract

Deep learning is widely used in various fields such as image processing

natural language processing

network mining and so on.However

it is vulnerable to malicious adversarial attacks and many defensive methods have been proposed accordingly.Most defense methods are attack-dependent and require defenders to generate massive adversarial examples in advance.The defense cost is high and it is difficult to resist black-box attacks.Some of these defenses even affect the recognition of normal examples.In addition

the current defense methods are mostly empirical

without certifiable theoretical support.Softmax activation transformation (SAT) was proposed in this paper

which was a light-weight and fast defense scheme against black-box attacks.SAT reactivates the output probability of the target model in the testing phase

and then it guarantees privacy of the probability information.As an attack-free defense

SAT not only avoids the burden of generating massive adversarial examples

but also realizes the advance defense of attacks.The activation of SAT is monotonic

so it will not affect the recognition of normal examples.During the activation process

a variable privacy protection transformation coefficient was designed to achieve dynamic defense.Above all

SAT is a certifiable defense that can derive the effectiveness and reliability of its defense based on softmax activation transformation.To evaluate the effectiveness of SAT

defense experiments against 9 attacks on MNIST

CIFAR10 and ImageNet datasets were conducted

and the average attack success rate was reduced from 87.06% to 5.94%.

关键词

Keywords

references

GOODFELLOW I , BENGIO Y , COURVILLE A . Deep learning [M ] . MIT Press , 2016 .

HELMSTAEDTER M , BRIGGMAN K L , TURAGA S C , et al . Connectomic reconstruction of the inner plexiform layer in the mouse retina [J ] . Nature , 2013 , 500 ( 7461 ): 168 - 174 .

XIONG H Y , ALIPANAHI B , LEE L J , et al . The human splicing code reveals new insights into the genetic determinants of disease [J ] . Science , 2015 , 347 ( 6218 ): 1254806 - 1254815 .

HINTON G E , DENG L , YU D , et al . Deep neural networks for acoustic modeling in speech recognition:The shared views of four research groups [J ] . IEEE Signal Processing Magazine , 2012 , 29 ( 6 ): 82 - 97 .

SUTSKEVER I , VINYALS O , LE Q V , et al . Sequence to sequence learning with neural networks [J ] . Advances in neural information processing systems , 2014 , 4 ( 2 ): 3104 - 3112 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E , et al . ImageNet classification with deep convolutional neural networks [J ] . Advances in Neural Information Processing Systems , 2012 , 2 : 1097 - 1105 .

DENG J , DONG W , SOCHER R , et al . ImageNet:a large-scale hierarchical image database [C ] // IEEE Conference on Computer Vision and Pattern Recognition . 2009 : 248 - 255 .

田野 , 项世军 . 基于 LBP 和多层 DCT 的人脸活体检测算法 [J ] . 计算机研究与发展 , 2018 , 55 ( 3 ): 643 - 650 .

TIAN Y , XIANG S J . LBP and multilayer DCT based anti-spoofing countermeasure in face liveness detection [J ] . Journal of Computer Research and Development , 2018 , 55 ( 3 ): 643 - 650 .

张蕊 , 李锦涛 . 基于深度学习的场景分割算法研究综述 [J ] . 计算机研究与发展 , 2020 , 57 ( 4 ): 859 - 875 .

ZHANG R , LI J T . A survey on algorithm research of scene parsing based on deep learning [J ] . Journal of Computer Research and De-velopment , 2020 , 57 ( 4 ): 859 - 875 .

NAJAFABADI M M , VILLANUSTRE F , KHOSHGOFTAAR T M , et al . Deep learning applications and challenges in big data analytics [J ] . Journal of Big Data , 2015 , 2 ( 1 ): 1 - 21 .

PAPERNOT N , MCDANIEL P , SINHA A , et al . Towards the science of security and privacy in machine learning [J ] . arXiv preprint arXiv:1611.03814 , 2016 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

SZEGEDY C , ZAREMBA W , SUTSKEVER I , et al . Intriguing properties of neural networks [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . 2013 : 7185 - 7193 .

GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [J ] . arXiv preprint arXiv:1412.6572 , 2014 .

DONG Y , LIAO F , PANG T , et al . Boosting adversarial attacks with momentum [C ] // /Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . 2018 : 9185 - 9193 .

MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O , et al . Universal adversarial perturbations [C ] // /Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2017 : 1765 - 1773 .

CARLINI N , WAGNER D . Towards evaluating the robustness of neural networks [C ] // /Proceedings of 2017 IEEE European Symposium on Security and Privacy . 2017 : 39 - 57 .

CHEN J , ZHENG H , XIONG H , et al . MAG-GAN:massive attack generator via GAN [J ] . Information Sciences , 2020 , 536 : 67 - 90 .

SU J , VARGAS D V , SAKURAI K . One pixel attack for fooling deep neural networks [J ] . IEEE Transactions on Evolutionary Computation , 2019 , 23 ( 5 ): 828 - 841 .

CHEN P Y , ZHANG H , SHARMA Y , et al . Zoo:zeroth order optimization based black-box attacks to deep neural networks without training substitute models [C ] // /Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security . 2017 : 15 - 26 .

CHEN J , SU M , SHEN S , et al . POBA-GA:perturbation optimized black-box adversarial attacks via genetic algorithm [J ] . Computers＆ Security , 2019 , 85 : 89 - 106 .

陈晋音 , 陈治清 , 郑海斌 , 等 . 基于粒子群优化的路牌识别模型的黑盒物理攻击方法 [J ] . 软件学报 , 2020 : 1 - 17 .

CHEN J Y , CHEN Z Q , ZHENG H B , et al . Black-box physical at-tack against road sign recognition model via PSO [J ] . Journal of Software , 2020 : 1 - 17 .

陈晋音 , 沈诗婧 , 苏蒙蒙 , 等 . 车牌识别系统的黑盒对抗攻击 [J ] . 自动化学报 , 2020 : 1 - 18 .

CHEN J Y , SHEN S J , SU M M , et al . Black-box adversarial attack on license plate recognition system [J ] . Acta Automatica Sinica , 2020 : 1 - 18 .

BRENDEL W , RAUBER J , BETHGE M . Decision-based adversarial attacks:reliable attacks against black-box machine learning models [J ] . arXiv preprint arXiv:1712.04248 , 2017 .

CHEN J , ZHENG H , CHEN R , et al . RCA-SOC:a novel adversarial defense by refocusing on critical areas and strengthening object contours [J ] . Computers ＆ Security , 2020 :101916.

AKHTAR N , MIAN A . Threat of adversarial attacks on deep learning in computer vision:a survey [J ] . IEEE Access , 2018 , 6 : 14410 - 14430 .

ALENAZY W M , ALQAHTANI A S . Gravitational search algorithm based optimized deep learning model with diverse set of features for facial expression recognition [J ] . Journal of Ambient Intelligence and Humanized Computing , 2021 , 12 ( 2 ): 1631 - 1646 .

TU C , TING P , CHEN P Y , et al . Autozoom:Autoencoder-based zeroth order optimization method for attacking black-box neural networks [J ] . arXiv preprint arXiv:1805.11770 , 2018 .

CHEN J , JORDAN M I . Boundary attack++:query-efficient decision-based adversarial attack [J ] . arXiv preprint arXiv:1904.02144 , 2019 .

PAPERNOT N , MCDANIEL P , GOODFELLOW I , et al . Practical black-box attacks against machine learning [C ] // /Proceedings of the 2017 ACM on Asia Conference on Computer and Communications . 2017 : 506 - 519 .

ZHANG X , WU D . On the vulnerability of CNN classifiers in EEG-based BCIs [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2019 , 27 ( 5 ): 814 - 825 .

SU J , VARGAS D V , SAKURAI K . One pixel attack for fooling deep neural networks [J ] . IEEE Transactions on Evolutionary Computation , 2019 , 23 ( 5 ): 828 - 841 .

ILYAS A , ENGSTROM L , ATHALYE A , et al . Black-box adversarial attacks with limited queries and information [C ] // /Proceedings of 35th International Conference on Machine Learning(ICML) . 2018 : 3392 - 3401 .

AKHTAR N , MIAN A . Threat of adversarial attacks on deep learning in computer vision:a survey [J ] . IEEE Access , 2018 , 6 : 14410 - 14430 .

GOODFELLOW IJ , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [J ] . arXiv preprint arXiv:1412.6572 .

KURAKIN A , GOODFELLOW I , BENGIO S , et al . Adversarial machine learning at scale [C ] // Proceedings of Computer Vision and Pattern Recognition(CVPR) . 2016 : 1 - 13 .

MADRY A , MAKELOV A , SCHMIDT L , et al . Towards deep learning models resistant to adversarial attacks [C ] // ICLR . 2018 .

MADRY A , MAKELOV A , SCHMIDT L , et al . Towards deep learning models resistant to adversarial attacks [J ] . arXiv preprint arXiv:1706.06083 , 2017 .

SHAFAHI A , NAJIBI M , GHIASI A , et al . Adversarial training for free! [J ] . arXiv preprint arXiv:1904.12843v1 .

XIE C , WANG J , ZHANG Z , et al . Adversarial examples for semantic segmentation and object detection [C ] // Proc of International Conference on Computer Vision(ICCV) . 2017 : 1 - 13 .

GUO C , RANA M , CISSE M , et al . Countering adversarial images using input transformations [J ] . arXiv preprint arXiv:1711.00117 , 2017 .

STRAUSS T , HANSELMANN M , JUNGINGER A , et al . Ensemble methods as a defense to adversarial perturbations against deep neural networks [J ] . arXiv preprint arXiv:1709.03423 , 2017 .

LIU X , CHENG M , ZHANG H , et al . Towards robust neural networks via random self-ensemble [C ] // /Proceedings of Lecture Notes in Computer Science , 2018 , 11211 : 381 - 397 .

Views

304

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Phishing website detection based on two-way feature extraction and adaptive fusion

Anomaly detection in DNS: an empirical study

FL-IoV: comprehensive intrusion detection method for internet of vehicles based on federated learning

Research on adversarial examples defense method based on multi-modal feature fusion

Multi-task adversarial attribution method based on hierarchical structure

Related Author

Guo Kai

Wang Heng

Ma Ziqiang

HOU Ronghao

HE Ming

KONG Kaichuan

CHEN Yong

ZUO Peng

Related Institution

School of Information Engineering, Ningxia University

China Internet Network Information Center

School of Cyber Science and Engineering, Jinan University

Beijing Unionpay Card Technology Co., Ltd., Shenzhen Branch

College of Computer Science and Engineering, Chongqing University of Technology

⁰