基于深度强化学习的恶意ELF文件检测对抗方法

孙贺; 张博成; 耿嘉炫; 吴迪; 王俊峰; 方智阳

doi:10.11959/j.issn.2096-109x.2024074

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的恶意ELF文件检测对抗方法

学术论文 | 更新时间：2024-12-20

- 基于深度强化学习的恶意ELF文件检测对抗方法
- Adversarial method for malicious ELF file detection based on deep reinforcement learning
- 网络与信息安全学报 2024年10卷第5期页码：152-162
- 作者机构：
  
  1.试验物理与计算数学重点实验室，北京 100094
  2.四川大学网络空间安全学院，四川成都 610207
  3.四川大学计算机学院，四川成都 610065
- 作者简介：
  
  [ "孙贺（1990- ），男，博士，试验物理与计算数学重点实验室助理研究员，主要研究方向为网络安全、二进制代码分析、漏洞挖掘。" ]
  [ "张博成（2003- ），男，四川蓬安人，主要研究方向为恶意代码分析。" ]
  [ "耿嘉炫（1994- ），男，福建福州人，四川大学博士生，主要研究方向为恶意软件分析、网络攻防对抗、机器学习安全。" ]
  [ "吴迪（1999- ），男，四川广安人，四川大学博士生，主要研究方向为软件逆向分析、软件安全、恶意软件分析。" ]
  [ "王俊峰（1976- ），男，安徽芜湖人，四川大学研究员、博士生导师，主要研究方向为空间信息网络、网络与信息安全。" ]
  [ "方智阳（1986- ），男，浙江永康人，四川大学副教授，主要研究方向为二进制安全、固件漏洞挖掘。" ]
- 基金信息：
  
  国家自然科学基金(U2133208);四川省科技厅重点研发项目(2023YFG0290)
- DOI：10.11959/j.issn.2096-109x.2024074
  中图分类号： TP309.5
- 收稿：2024-04-30，
  
  修回：2024-07-15，
  
  纸质出版：2024-10-25
- 稿件说明：
移动端阅览
孙贺, 张博成, 耿嘉炫, 等. 基于深度强化学习的恶意ELF文件检测对抗方法[J]. 网络与信息安全学报, 2024, 10(5): 152-162.

Format: SUN H, ZHANG B C, GENG J X, et al. Adversarial method for malicious ELF file detection based on deep reinforcement learning[J]. Chinese Journal of Network and Information Security, 2024, 10(5): 152-162.
孙贺, 张博成, 耿嘉炫, 等. 基于深度强化学习的恶意ELF文件检测对抗方法[J]. 网络与信息安全学报, 2024, 10(5): 152-162. DOI： 10.11959/j.issn.2096-109x.2024074.

Format: SUN H, ZHANG B C, GENG J X, et al. Adversarial method for malicious ELF file detection based on deep reinforcement learning[J]. Chinese Journal of Network and Information Security, 2024, 10(5): 152-162. DOI： 10.11959/j.issn.2096-109x.2024074.

摘要

近年来，基于深度学习的恶意可执行与可链接格式（executable and linkable format，ELF）文件检测研究取得了显著进展。同时，关于模型对抗性攻击的研究也得到了广泛关注，攻击者可以通过生成对抗样本误导神经网络，使恶意软件被错误归类为良性软件，从而逃逸检测。尽管已有多种生成对抗样本的方法被提出，但它们通常不适合对ELF文件进行修改，或缺乏在不同检测模型之间迁移的能力。为克服现有方法的局限性，提出了一种基于深度强化学习的恶意ELF文件检测对抗方法，该方法通过构造目标检测模型的最优扰动字节序列，能够在保留ELF文件原有功能的前提下生成对抗样本，而无须依赖目标模型的内部实现细节。实验结果表明，该方法生成的对抗样本针对目标检测模型的逃逸成功率达到76.80%，并能通过对抗训练提升目标检测模型的鲁棒性。

Abstract

In recent years

research on detecting malicious executable and linkable format (ELF) files based on deep learning had made significant progress. At the same time

adversarial attacks on models had also gained widespread attention. Attackers could generate adversarial examples to mislead neural networks

causing malicious software to be misclassified as benign

thereby evading detection. Although various methods for generating adversarial examples had been proposed

they were often not suitable for modifying ELF files or lacked the ability to transfer across different detection models. To overcome the limitations of existing methods

an adversarial example generation method based on deep reinforcement learning was proposed. This method generated adversarial examples by constructing optimal perturbation bytes for the target detection model while preserving the original functionality of the ELF files

without relying on the internal details of the target model. The experimental results showed that the adversarial examples generated by this method achieved a 76.80% success rate in evading the target detection model

and could enhance the robustness of the model through adversarial training.

关键词

Keywords

references

GALANTE L , BOTACIN M , GRÉGIO A , et al . Malicious Linux binaries: a landscape [C ] // Proceedings of the Anais Estendidos do XVIII Simpósio Brasileiro de Segurança da Informação e de Sistemas Computacionais (SBSeg Estendido 2018). 2018 : 213 - 222 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C ] // Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2016 : 770 - 778 .

AMODEI D , ANUBHAI R , BATTENBERG E , et al . Deep speech 2: end-to-end speech recognition in English and mandarin [EB ] . arXiv preprint arxiv: 1512.02595 , 2015 .

DEVLIN J , CHANG M W , LEE K , et al . BERT: pre-training of deep bidirectional transformers for language understanding [EB ] . arXiv preprint arXiv: 1810.04805 , 2018 .

JEON J , JEONG B , BAEK S , et al . Hybrid malware detection based on Bi-LSTM and SPP-net for smart IoT [J ] . IEEE Transactions on Industrial Informatics , 2022 , 18 ( 7 ): 4830 - 4837 .

WANG H R , HE H , ZHANG W Z . Demadroid: object reference graph-based malware detection in android [J ] . Security and Communication Networks , 2018 : 7064131 .

CORUM A , JENKINS D , ZHENG J . Robust PDF malware detection with image visualization and processing techniques [C ] // Proceedings of the 2019 2nd International Conference on Data Intelligence and Security (ICDIS) . Piscataway : IEEE Press , 2019 : 108 - 114 .

COULL S E , GARDNER C . Activation analysis of a byte-based deep neural network for malware classification [C ] // Proceedings of the 2019 IEEE Security and Privacy Workshops (SPW) . Piscataway : IEEE Press , 2019 : 21 - 27 .

KOLOSNJAJI B , DEMONTIS A , BIGGIO B , et al . Adversarial malware binaries: evading deep learning for malware detection in executables [C ] // Proceedings of the 2018 26th European Signal Processing Conference (EUSIPCO) . Piscataway : IEEE Press , 2018 : 533 - 537 .

DEMETRIO L , BIGGIO B , LAGORIO G , et al . Explaining vulnerabilities of deep learning to adversarial malware binaries [EB ] . arXiv preprint arXiv: 1901.03583 , 2019 .

KREUK F , BARAK A , AVIV-REUVEN S , et al . Deceiving end-to-end deep learning malware detectors using adversarial examples [EB ] . arXiv preprint arXiv: 1802.04528 , 2018 .

DEMETRIO L , COULL S E , BIGGIO B , et al . Adversarial examples [J ] . ACM Transactions on Privacy and Security , 2021 , 24 ( 4 ): 1 - 31 .

RAFF E , BARKER J , SYLVESTER J , et al . Malware detection by eating a whole exe [C ] // Proceedings of the Workshops at the Thirty-Second AAAI Conference on Artificial Intelligence . 2018 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Playing atari with deep reinforcement learning [EB ] . arXiv preprint arXiv: 1312.5602 , 2013 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

SZEGEDY C , ZAREMBA W , SUTSKEVER I , et al . Intriguing properties of neural networks [EB ] . arXiv preprint arXiv: 1312.6199 , 2013 .

GOODFELLOW I J , SHLENS J , SZEGEDY C , et al . Explaining and harnessing adversarial examples [J ] . arXiv preprint arXiv: 1412.6572 , 2014 .

AL-DUJAILI A , HUANG A , HEMBERG E , et al . Adversarial deep learning for robust detection of binary encoded malware [C ] // Proceedings of the 2018 IEEE Security and Privacy Workshops (SPW) . Piscataway : IEEE Press , 2018 : 76 - 82 .

DING Y X , SHAO M M , NIE C , et al . An efficient method for generating adversarial malware samples [J ] . Electronics , 2022 , 11 ( 1 ): 154 .

KOSIKOWSKI A , CHO D , NINAN M , et al . EvilELF: evasion attacks on deep-learning malware detection over ELF files [C ] // Proceedings of the 2023 International Conference on Machine Learning and Applications (ICMLA) . Piscataway : IEEE Press , 2023 : 1702 - 1709 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于模仿学习的深度强化学习训练数据推断攻击

基于分布散度的自适应模糊测试优化方法

基于注意力与门控机制的多特征融合恶意软件检测方法

基于深度强化学习的高性能导向性模糊测试方案

基于深度神经网络的Android恶意软件检测方法