Visual explanation method for reversible neural networks

Xinying MU; Bingbing SONG; Fanxiao LI; Yisen ZHENG; Wei ZHOU; Yunyun DONG

doi:10.11959/j.issn.2096-109x.2023090

您当前的位置：

首页 >

文章列表页 >

Visual explanation method for reversible neural networks

Papers | 更新时间：2024-06-03

- Visual explanation method for reversible neural networks
- Chinese Journal of Network and Information Security Vol. 9, Issue 6, Pages: 154-165(2023)
- 作者机构：
  
  1. 云南大学国家示范性软件学院，云南昆明 650000
  2. 云南大学信息学院，云南昆明 650000
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2023090
  CLC： TP393
- Online First：2023-12，
  
  Published：25 December 2023
- 稿件说明：
移动端阅览
Xinying MU, Bingbing SONG, Fanxiao LI, et al. Visual explanation method for reversible neural networks[J]. Chinese Journal of Network and Information Security, 2023, 9(6): 154-165.
DOI：

Xinying MU, Bingbing SONG, Fanxiao LI, et al. Visual explanation method for reversible neural networks[J]. Chinese Journal of Network and Information Security, 2023, 9(6): 154-165. DOI： 10.11959/j.issn.2096-109x.2023090.

摘要

为了更好地理解深度神经网络（DNN，deep neural network）在应用过程中出现的决策依据未知以及容易受到对抗攻击等安全问题，模型可解释性受到广泛关注。虽然越来越多的学者针对传统深度神经网络的可解释性进行了研究，但对可逆神经网络的运行机制和可解释性的探索还存在不足，且现有针对传统深度神经网络的解释方法不适用于可逆神经网络，存在噪声大、梯度饱和等问题。因此，提出一种针对可逆神经网络的可视化解释方法，其基于类激活映射机制，利用可逆神经网络的可逆特性探索特征图与输入图像之间的区域对应关系，使得区域特征图的分类权重可映射到输入图像的对应区域，得到输入图像每个区域对模型决策的重要程度，从而生成模型决策依据。在通用数据集上，将所提方法与其他解释方法进行实验比较，所提方法取得了更集中的视觉效果，在识别任务中，相较于次优方法平均下降（AD，average drop）指标提升7.80%，平均上升（AI，average increase）指标提升6.05%，热值最大点的定位水平达到82.00%，同时，所提方法可以对传统深度神经网络进行解释且其良好的扩展性可以提高其他方法对可逆神经网络的解释性能。另外，在对抗攻击解析实验中发现，对抗攻击使得模型的决策依据发生改变，体现在模型的关注区域发生错位，这有助于探究对抗攻击的运行机制。

Abstract

The issue of model explainability has gained significant attention in understanding the vulnerabilities and anonymous decision-making processes inherent in deep neural networks (DNN).While there has been considerable research on explainability for traditional DNN

there is a lack of exploration on the operation mechanism and explainability of reversible neural networks (RevNN).Additionally

the existing explanation methods for traditional DNN are not suitable for RevNN and suffer from issues such as excessive noise and gradient saturation.To address these limitations

a visual explanation method called visual explanation method for reversible neural network (VERN) was proposed for RevNN.VERN leverages the reversible property of RevNN and is based on the class-activation mapping mechanism.The correspondence between the feature map and the input image was explored by VERN

allowing for the mapping of classification weights of regional feature maps to the corresponding regions of the input image.The importance of each region for model decision-making was revealed through this process

which generates a basis for model decision-making.Experimental comparisons with other explanation methods on generalized datasets demonstrate that VERN achieves a more focused visual effect

surpassing suboptimal methods with up to 7.80% improvement in average drop (AD) metrics and up to 6.05% improvement in average increase (AI) metrics in recognition tasks.VERN also exhibits an 82.00% level of localization for the maximum point of the heat value.Furthermore

VERN can be applied to explain traditional DNN and exhibits good scalability

improving the performance of other methods in explaining RevNN.Furthermore

through adversarial attack analysis experiments

it is observed that adversarial attacks alter the decision basis of the model.This is reflected in the misalignment of the model’s attention regions

thereby aiding in the exploration of the operation mechanism of adversarial attacks.

关键词

Keywords

references

谭清尹 , 曾颖明 , 韩叶 , 等 . 神经网络后门攻击研究 [J ] . 网络与信息安全学报 , 2021 , 7 ( 3 ): 46 - 58 .

TAN Q Y , ZENG Y M , HAN Y , et al . Survey on backdoor attacks targeted on neural network [J ] . Chinese Journal of Network and Information Security , 2021 , 7 ( 3 ): 46 - 58 .

杨朋波 , 桑基韬 , 张彪 , 等 . 面向图像分类的深度模型可解释性研究综述 [J ] . 软件学报 , 2023 , 34 ( 1 ): 230 - 254 .

YANG P B , SANG J T , ZHANG B , et al . Survey on interpretability of deep models for image classification [J ] . Journal of Software , 2023 , 34 ( 1 ): 230 - 254 .

FANG Z , KUANG K , LIN Y , et al . Concept-based explanation for fine-grained images and its application in infectious keratitis classification [C ] // Proceedings of the 28th ACM International Conference on Multimedia . 2020 : 700 - 708 .

化盈盈 , 张岱墀 , 葛仕明 . 深度学习模型可解释性的研究进展 [J ] . 信息安全学报 , 2020 , 5 ( 3 ): 1 - 12 .

HUA Y Y , ZHANG D X , GE S M , et al . Research progress in the interpretability of deep learning models [J ] . Journal of Cyber Security , 2020 , 5 ( 3 ): 1 - 12 .

纪守领 , 李进锋 , 杜天宇 , 等 . 机器学习模型可解释性方法、应用与安全研究综述 [J ] . 计算机研究与发展 , 2019 , 56 ( 10 ): 2071 - 2096 .

JI S L , LI J F , DU T Y , et al . Survey on the techniques,applications and security of machine learning interpretablity [J ] . Journal of Computer Research and Development , 2019 , 56 ( 10 ): 2071 - 2096 .

ZEILER M D , FERGUS R . Visualizing and understanding convolutional networks [C ] // Proceedings of European Conference on Computer Vision–ECCV 2014 . 2014 : 818 - 833 .

AGARWAL C , NGUYEN A . Explaining image classifiers by removing input features using generative models [C ] // Proceedings of the Asian Conference on Computer Vision . 2020 .

FONG R C , VEDALDI A . Interpretable explanations of black boxes by meaningful perturbation [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2017 : 3429 - 3437 .

SELVARAJU R R , COGSWELL M , DAS A , et al . Grad-CAM:visual explanations from deep networks via gradient-based localization [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2017 : 618 - 626 .

CHATTOPADHAY A , SARKAR A , HOWLADER P , et al . Grad-CAM++:generalized gradient-based visual explanations for deep convolutional networks [C ] // 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) . 2018 : 839 - 847 .

WANG H , WANG Z , DU M , et al . Score-CAM:score-weighted visual explanations for convolutional neural networks [C ] // Proceedings of the IEEE/CVF Conference on Computer vision and Pattern Recognition Workshops . 2020 : 24 - 25 .

JACOBSEN J H , ARNOLD W M S , EDOUARD O . I-RevNet:deep invertible networks [C ] // Proceedings of International Conference on Learning Representations . 2018 .

JING J , DENG X , XU M , et al . Hinet:deep image hiding by invertible network [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . 2021 : 4733 - 4742 .

LUGMAYR A , DANELLJAN M , VAN GOOL L , et al . Srflow:learning the super-resolution space with normalizing flow [C ] // Proceedings of Computer Vision–ECCV 2020 . 2020 : 715 - 732 .

ZHOU B , KHOSLA A , LAPEDRIZA A , et al . Learning deep features for discriminative localization [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 2921 - 2929 .

SIMONYAN K , VEDALDI A , ZISSERMAN A . Deep inside convolutional networks:visualising image classification models and saliency maps [J ] . arXiv Preprint arXiv:1312.6034 , 2013 .

SMILKOV D , THORAT N , KIM B , et al . Smoothgrad:removing noise by adding noise [J ] . arXiv Preprint arXiv:1706.03825 , 2017 .

SPRINGENBERG J T , DOSOVITSKIY A , BROX T , et al . Striving for simplicity:the all convolutional net [J ] . arXiv Preprint arXiv:1412.6806 , 2014 .

SUNDARARAJAN M , TALY A , YAN Q . Axiomatic attribution for deep networks [C ] // International Conference on Machine Learning . 2017 : 3319 - 3328 .

PETSIUK V , DAS A , SAENKO K . Rise:randomized input sampling for explanation of black-box models [J ] . arXiv Preprint arXiv:1806.07421 , 2018 .

GOMEZ A N , REN M , URTASUN R , et al . The reversible residual network:backpropagation without storing activations [J ] . Advances in Neural Information Processing Systems , 2017 ,30.

RAMASWAMY H G . Ablation-cam:visual explanations for deep convolutional network via gradient-free localization [C ] // Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision . 2020 : 983 - 991 .

KRIZHEVSKY A , HINTON G . Learning multiple layers of features from tiny images [R ] . 2009 .

DENG J , DONG W , SOCHER R , et al . ImageNet:a large-scale hierarchical image database [C ] // Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition . 2009 : 248 - 255 .

MADRY A , MAKELOV A , SCHMIDT L , et al . Towards deep learning models resistant to adversarial attacks [J ] . arXiv Preprint arXiv:1706.06083 , 2017 .

DONG Y , LIAO F , PANG T , et al . Boosting adversarial attacks with momentum [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 : 9185 - 9193 .

TRAMÈR F , KURAKIN A , PAPERNOT N , et al . Ensemble adversarial training:Attacks and defenses [J ] . arXiv Preprint arXiv:1705.07204 , 2017 .

KHAKZAR A , KHORSANDI P , NOBAHARI R , et al . Do explanations explain? Model knows best [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2022 : 10244 - 10253 .

DABKOWSKI P , GAL Y . Real time image saliency for black box classifiers [C ] // Advances in Neural Information Processing Systems . 2017 ,30.

Views

229

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on construction technology of artificial intelligence security knowledge graph

Survey of split learning data privacy

Privacy leakage risk assessment for reversible neural network

Survey on backdoor attacks targeted on neural network

ChinaVis Data Challenge from 2015 to 2017

Related Author

Ling YU

Bo CHEN

Xiaochen SHEN

Yinhui GE

JIN Hai

XU Peng

HU Pingyi

FU Jiayun

Related Institution

School of Computer and Electronic Information/School of Artificial Intelligence, Nanjing Normal University

School of Mathematical Science, Nanjing Normal University

Services Computing Technology and System Lab, National Engineering Research Center for Big Data Technology

School of Cyber Science and Engineering, Huazhong University of Science and Technology

School of Computer Science and Engineering, Huazhong University of Science and Technology

⁰