基于剪枝技术和鲁棒蒸馏融合的轻量对抗攻击防御方法

王滨; 李思敏; 钱亚冠; 张君; 李超豪; 朱晨鸣; 张鸿飞

doi:10.11959/j.issn.2096-109x.2022074

您当前的位置：

首页 >

文章列表页 >

基于剪枝技术和鲁棒蒸馏融合的轻量对抗攻击防御方法

学术论文 | 更新时间：2024-06-03

- 基于剪枝技术和鲁棒蒸馏融合的轻量对抗攻击防御方法
- Lightweight defense mechanism against adversarial attacks via adaptive pruning and robust distillation
- 网络与信息安全学报 2022年8卷第6期页码：102-109
- 作者机构：
  
  1. 浙江科技学院，浙江杭州 310023
  2. 浙江省多维感知技术应用与安全重点实验室，浙江杭州 310052
  3. 浙江省电子信息产品检验研究院，浙江杭州 310007
- 作者简介：
  
  [ "王滨（1978- ），男，山东泗水人，博士，浙江省多维感知技术应用与安全重点实验室研究员、博士生导师，主要研究方向为物联网安全、人工智能安全、密码学" ]
  [ "李思敏（1997- ），男，浙江义乌人，浙江科技学院硕士生，主要研究方向为深度学习、人工智能安全和模型压缩" ]
  [ "钱亚冠（1976- ），男，浙江嵊州人，博士，浙江科技学院理学院教授，主要研究方向为深度学习、人工智能安全" ]
  [ "张君（1980- ），女，河南内乡人，浙江省信息化发展中心高级工程师，主要研究方向为网络与信息安全" ]
  [ "李超豪（1995- ），男，浙江温州人，博士，浙江省多维感知技术应用与安全重点实验室副主任，主要研究方向为物联网安全、人工智能安全、感知对抗安全、数据隐私保护" ]
  [ "朱晨鸣（1981- ），男，浙江杭州人，浙江省电子信息产品检验研究院高级工程师，主要研究方向为网络与信息安全" ]
  [ "张鸿飞（1984- ），男，浙江嘉兴人，浙江省电子信息产品检验研究院工程师，主要研究方向为网络与信息安全" ]
- 基金信息：
  
  国家自然科学基金;The National Natural Science Foundation of China(92167203);浙江省自然科学基金(LZ22F020007)
- DOI：10.11959/j.issn.2096-109x.2022074
  中图分类号： TP393
- 网络首发：2022-12，
  
  纸质出版：2022-12-15
- 稿件说明：
移动端阅览
王滨, 李思敏, 钱亚冠, 等. 基于剪枝技术和鲁棒蒸馏融合的轻量对抗攻击防御方法[J]. 网络与信息安全学报, 2022,8(6):102-109.

Bin WANG, Simin LI, Yaguan QIAN, et al. Lightweight defense mechanism against adversarial attacks via adaptive pruning and robust distillation[J]. Chinese Journal of Network and Information Security, 2022, 8(6): 102-109.
王滨, 李思敏, 钱亚冠, 等. 基于剪枝技术和鲁棒蒸馏融合的轻量对抗攻击防御方法[J]. 网络与信息安全学报, 2022,8(6):102-109. DOI： 10.11959/j.issn.2096-109x.2022074.

Bin WANG, Simin LI, Yaguan QIAN, et al. Lightweight defense mechanism against adversarial attacks via adaptive pruning and robust distillation[J]. Chinese Journal of Network and Information Security, 2022, 8(6): 102-109. DOI： 10.11959/j.issn.2096-109x.2022074.

摘要

对抗训练是一类常用的对抗攻击防御方法，其通过将对抗样本纳入训练过程，从而有效抵御对抗攻击。然而，对抗训练模型的鲁棒性通常依赖于网络容量的提升，即对抗训练所获得的网络为防御对抗攻击而大幅提升网络的模型容量，对其可用性造成较大约束。因此，如何在保证对抗训练模型鲁棒性的同时，降低模型容量，提出轻量对抗攻击防御方法是一大挑战。为解决以上问题，提出一种基于剪枝技术和鲁棒蒸馏融合的轻量对抗攻击防御方法。该方法以对抗鲁棒准确率为优化条件，在对预训练的鲁棒对抗模型进行分层自适应剪枝压缩的基础上，再对剪枝后的网络进行基于数据过滤的鲁棒蒸馏，实现鲁棒对抗训练模型的有效压缩，降低其模型容量。在CIFAR-10和CIFAR-100数据集上对所提出的方法进行性能验证与对比实验，实验结果表明，在相同 TRADES 对抗训练下，所提出的分层自适应剪枝技术相较于现有剪枝技术，其剪枝所得到的网络结构在多种 FLOPs 下均表现出更强的鲁棒性。此外，基于剪枝技术和鲁棒蒸馏融合的轻量对抗攻击防御方法相较于其他鲁棒蒸馏方法表现出更高的对抗鲁棒准确率。因此，实验结果证明所提方法在降低对抗训练模型容量的同时，相较于现有方法具有更强的鲁棒性，提升了对抗训练模型在物联网边缘计算环境的适用性。

Abstract

Adversarial training is one of the commonly used defense methods against adversarial attacks

by incorporating adversarial samples into the training process.However

the effectiveness of adversarial training heavily relied on the size of the trained model.Specially

the size of trained models generated by the adversarial training will significantly increase for defending against adversarial attacks.This imposes constraints on the usability of adversarial training

especially in a resource-constraint environment.Thus

how to reduce the model size while ensuring the robustness of the trained model is a challenge.To address the above issues

a lightweight defense mechanism was proposed against adversarial attacks

with adaptive pruning and robust distillation.A hierarchically adaptive pruning method was applied to the model generated by adversarial training in advance.Then the trained model was further compressed by a modified robust distillation method.Experimental results on CIFAR-10 and CIFAR-100 datasets showed that our hierarchically adaptive pruning method presented stronger robustness under various FLOP than the existing pruning methods.Moreover

the fusion of pruning and robust distillation presented higher robustness than the state-of-art robust distillation methods.Therefore

the experimental results prove that the proposed method can improve the usability of the adversarial training in the IoT edge computing environment.

关键词

Keywords

references

郑远攀 , 李广阳 , 李晔 . 深度学习在图像识别中的应用研究综述 [J ] . 计算机工程与应用 , 2019 , 55 ( 12 ): 20 - 36 .

ZHENG Y P , LI G Y , LI Y . Survey of application of deep learning in image recognition [J ] . Computer Engineering and Applications , 2019 , 55 ( 12 ): 20 - 36 .

鱼昆 , 张绍阳 , 侯佳正 , 等 . 语音识别及端到端技术现状及展望 [J ] . 计算机系统应用 , 2021 , 30 ( 3 ): 14 - 23 .

YU K , ZHANG S Y , HOU J Z , et al . Survey of speech recognition and end-to-end techniques [J ] . Computer Systems ＆ Applications , 2021 , 30 ( 3 ): 14 - 23 .

王睿怡 , 罗森林 , 吴舟婷 , 等 . 深度学习在汉语语义分析的应用与发展趋势 [J ] . 计算机技术与发展 , 2019 , 29 ( 9 ): 110 - 116 .

WANG R Y , LUO S L , WU Z T , et al . Application and development trend of deep learning in Chinese semantic analysis [J ] . Computer Technology and Development , 2019 , 29 ( 9 ): 110 - 116 .

SZEGEDY C , ZAREMBA W , SUTSKEVER I , et al . Intriguing properties of neural networks [C ] // Proceedings of 2nd International Conference on Learning Representations (ICLR 2014) . 2014 .

CARLINI N , WAGNER D . Towards evaluating the robustness of neural networks [C ] // Proceedings of 2017 IEEE Symposium on Security and Privacy (SP) . 2017 .

SONG D , EYKHOLT K , EVTIMOV I , et al . Physical adversarial examples for object detectors [C ] // 12th USENIX Workshop on offensive technologies (WOOT 18) . 2018 .

CARLINI N , WAGNER D . Audio adversarial examples:targeted attacks on speech-to-text [C ] // 2018 IEEE Security and Privacy Workshops (SPW) . 2018 : 1 - 7 .

GOODFELLOW I , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [C ] // Proceedings of 3rd International Conference on Learning Representations (ICLR 2015) . 2015 .

MADRY A , MAKELOV A , SCHMIDT L , et al . Towards deep learning models resistant to adversarial attacks [C ] // Proceedings of 6th International Conference on Learning Representations (ICLR 2018) . 2018 .

GOWAL S , QIN C , UESATO J , et al . Uncovering the limits of adversarial training against norm-bounded adversarial examples [J ] . arXiv:2010.03593 , 2020 .

GOU J , YU B , MAYBANK S J , et al . Knowledge distillation:a survey [J ] . International Journal of Computer Vision , 2021 , 129 ( 6 ): 1789 - 1819 .

PAPERNOT N , MCDANIEL P , WU X , et al . Distillation as a defense to adversarial perturbations against deep neural networks [C ] // IEEE Symposium on Security and Privacy , 2016 : 582 - 597 .

GOLDBLUM M , FOWL L , FEIZI S , et al . Adversarially robust distillation [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . 2020 : 3996 - 4003 .

ZI B , ZHAO S , MA X , et al . Revisiting adversarial robustness distillation:robust soft labels make student better [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision , 2021 : 16423 - 16432 .

BAI T , LUO J , ZHAO J , et al . Recent advances in adversarial training for adversarial robustness [C ] // Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence , 2021 : 4312 - 4321 .

ZHANG H , YU Y , JIAO J , et al . Theoretically principled trade-off between robustness and accuracy [C ] // International Conference on Machine Learning . 2019 : 7472 - 7482 .

ALAYRAC J B , UESATO J , HUANG P S , et al . Are labels required for improving adversarial robustness [C ] // Advances in Neural Information Processing Systems . 2019 : 12192 - 12202 .

CARMON Y , RAGHUNATHAN A , SCHMIDT L , et al . Unlabeled data improves adversarial robustness [C ] // Advances in Neural Information Processing Systems . 2019 : 11190 - 11201 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [J ] . arXiv:1409.1556 , 2014 .

HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition [C ] // Proceedings of 2016 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2016 : 770 - 778 .

CROCE F , HEIN M . Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks [C ] // Proceedings of the 37th International Conference on Machine Learning . 2020 : 2206 - 2216 .

LI H , KADAV A , DURDANOVIC I , et al . Pruning filters for efficient convnets [C ] // Proceedings of 5th International Conference on Learning Representations (ICLR 2017) . 2017 .

LIU Z , LI J , Shen Z , et al . Learning efficient convolutional networks through network slimming [C ] // Proceedings of 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2017 : 2755 - 2763 .

SUI Y , YIN M , XIE Y , et al . CHIP:channel Independence-based pruning for compact neural networks [C ] // Advances in Neural Information Processing Systems . 2021 : 24604 - 24616 .

浏览量

494

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

深度学习中的对抗攻击与防御