Self-adaptive fuzzing optimization method based on distribution divergence

XU Hang; JI Jiangan; MA Zheyu; ZHANG Chao

doi:10.11959/j.issn.2096-109x.2024079

您当前的位置：

首页 >

文章列表页 >

Self-adaptive fuzzing optimization method based on distribution divergence

Papers | 更新时间：2025-02-06

- Self-adaptive fuzzing optimization method based on distribution divergence
- Chinese Journal of Network and Information Security Vol. 10, Issue 6, Pages: 37-58(2024)
- 作者机构：
  
  1.网络空间安全教育部重点实验室，河南郑州 450001
  2.清华大学网络科学与网络空间研究院，北京 100080
- 作者简介：
- 基金信息：
  
  The National Key Research and Development Program of China(2021YFB2701000)
- DOI：10.11959/j.issn.2096-109x.2024079
  CLC： TP311.5
- Received：11 July 2024，
  
  Revised：2024-11-21，
  
  Published：25 December 2024
- 稿件说明：
移动端阅览
许航, 计江安, 马哲宇, 等. 基于分布散度的自适应模糊测试优化方法[J]. 网络与信息安全学报, 2024, 10(6): 37-58.

XU H, JI J A, MA Z Y, et al. Self-adaptive fuzzing optimization method based on distribution divergence[J]. Chinese Journal of Network and Information Security, 2024, 10(6): 37-58.
许航, 计江安, 马哲宇, 等. 基于分布散度的自适应模糊测试优化方法[J]. 网络与信息安全学报, 2024, 10(6): 37-58. DOI： 10.11959/j.issn.2096-109x.2024079.

XU H, JI J A, MA Z Y, et al. Self-adaptive fuzzing optimization method based on distribution divergence[J]. Chinese Journal of Network and Information Security, 2024, 10(6): 37-58. DOI： 10.11959/j.issn.2096-109x.2024079.

摘要

为了提高覆盖引导的模糊测试的性能，提出一种利用分布散度和深度强化学习模型自适应模糊测试的优化方法。基于过程间控制流图构建过程间比较流图，以刻画被测程序分支判断变量对应的空间随机场，并使用蒙特卡洛方法提取模糊测试变异策略产生的随机场分布特征；构建深度图卷积神经网络，以提取过程

间比较流图的特征嵌入，并将该神经网络作为深度强化学习的深度Q网络；基于双重深度Q网络模型建立在线深度强化学习模型，从而训练智能代理以优化模糊测试变异策略。该深度强化学习模型利用种子文件及相关区块对应的随机场分布特征定义状态，将种子文件重点变异区块的选择定义为动作，并将动作前后随机场近似分布的分布散度定义为奖励。针对该模糊测试优化方法实现了原型系统，并对其开展了多轮次的持续24 h的评估。实验结果表明，在测试集FuzzBench上，该原型系统代码覆盖速度和总体覆盖明显优于基线模糊测试器AFL++与Havoc

MAB

，并在多数基准测试目标上优于CmpLog；在测试集Magma上，该原型系统在基准测试目标openssl、libxml和sqlite3上具备更强的漏洞触发能力。

Abstract

To improve the performance of coverage-guided fuzzing

a method for self-adaptive optimization of fuzzing using distribution divergence and a deep reinforcement learning model was proposed. An interprocedural comparison flow graph was first constructed based on the interprocedural control flow graph to characterize the spatial random field corresponding to the branch condition variables of the program under test

and the distribution features of the random field generated by a fuzzing mutation strategy were extracted using the Monte Carlo method. Then

a deep graph convolutional neural network was constructed to extract the feature embeddings of the interprocedural comparison flow graph

and this neural network was used as the deep Q-network for deep reinforcement learning. Finally

an online deep reinforcement learning model was established based on the dual deep Q-network model

and an intelligent agent was trained to optimize the fuzzing mutation strategy. This deep reinforcement learning model defined its state using the random field distribution features corresponding to the seed file and the associated blocks. The selection for the focused mutation block of a seed file was defined as an action

and the distribution divergence of the approximate distributions of the random fields before and after the action was defined as the reward. A prototype was implemented for this fuzzing optimization method

and multiple rounds of up to 24 hours of evaluation were carried out on this prototype. The experimental results show that on the benchmark FuzzBench

the code coverage speed and overall coverage achieved by the prototype are significantly better than those of the baseline fuzzer AFL++ and HavocMAB

and better results are achieved on most benchmarks compared to CmpLog. On the benchmark Magma

stronger vulnerability triggering capability is demonstrated by the prototype on the benchmarks openssl

libxml

and sqlite3.

关键词

Keywords

references

BÖHME M , PHAM V T , ROYCHOUDHURY A . Coverage-based greybox fuzzing as markov chain [C ] // Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security . 2016 : 1032 - 1043 .

RAWAT S , JAIN V , KUMAR A , et al . VUzzer: application-aware Evolutionary Fuzzing [C ] // NDSS : 2017 : 1 - 14 .

GAN S , ZHANG C , QIN X , et al . Collafl: path sensitive fuzzing [C ] // 2018 IEEE Symposium on Security and Privacy (SP) . 2018 : 679 - 696 .

LEMIEUX C , SEN K . FairFuzz: a targeted mutation strategy for increasing greybox fuzz testing coverage [C ] // Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering . 2018 : 475 - 485 .

YUE T , WANG P , TANG Y , et al . EcoFuzz: adaptive energy-saving greybox fuzzing as a variant of the adversarial multi-armed bandit [C ] // 29th USENIX Security Symposium (USENIX Security 20) . USENIX Association , 2020 : 2307 - 2324 .

MANTOVANI A , FIORALDI A , BALZAROTTI D . Fuzzing with data dependency information [C ] // 7th IEEE European Symposium on Security and Privacy（EuroS&P 2022）. 2022 : 286 - 302 .

ASCHERMANN C , SCHUMILO S , BLAZYTKO T , et al . REDQUEEN: fuzzing with input-to-state correspondence [C ] // 26th Annual Network and Distributed System Security Symposium（NDSS 2019）. 2019 . 2019 .

FIORALDI A , D’ELIA D C , COPPA E . WEIZZ: automatic grey-box fuzzing for structured binary formats [C ] // Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis . 2020 : 1 - 13 .

METZMAN J , SZEKERES L , SIMON L , et al . Fuzzbench: an open fuzzer benchmarking platform and service [C ] // Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . 2021 : 1393 - 1403 .

HAZIMEH A , HERRERA A , PAYER M . Magma: A ground-truth fuzzing benchmark [C ] // Proc. ACM Meas. Anal. Comput. Syst. , 2020 , 4(3) .

WANG J , SONG C , YIN H . Reinforcement learning-based hierarchical seed scheduling for greybox fuzzing [C ] // 28th Annual Network and Distributed System Security Symposium(NDSS 2021). 2021 .

SHE D , SHAH A , JANA S . Effective seed scheduling for fuzzing with graph centrality analysis [C ] // 43rd IEEE Symposium on Security and Privacy(SP 2022). 2022 : 2194 - 2211 .

BÖHME M , MANÈS V J M , CHA S K . Boosting fuzzer efficiency: an information theoretic perspective [C ] // 28th ACM Joint European Software Engineering Conferenceand Symposium on the Foundations of Software Engineering . 2020 : 678 - 689 .

ZALEWSKI M . American fuzzy lop, 2017 [EB ] .

SCARSELLI F , GORI M , TSOI A C , et al . The graph neural network model [J ] . IEEE Trans Neural Networks , 2009 , 20 ( 1 ): 61 - 80 .

SUTTON R S , BARTO A G . Reinforcement learning - an introduction [M ] . MIT Press , 1998 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 : 529 - 533 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Playing atari with deep reinforcement learning [J ] . CoRR , 2013 , abs/1312.5.

FIORALDI A , MAIER D C , EISSFELDT H , et al . AFL++ : combining incremental steps of fuzzing research [C ] // 14th USENIX Workshop on Offensive Technologies , WOOT 2020 . 2020 .

LEE J , LEE I , KANG J . Self-attention graph pooling [C ] // Proceedings of the 36th International Conference on Machine Learning . 2019 : 3734 - 3743 .

LATTNER C , ADVE V . LLVM: a compilation framework for lifelong program analysis & transformation [C ] // International Symposium on Code Generation and Optimization(CGO 2004). 2004 : 75 - 86 .

HAGBERG A , SWART P , S CHULT D . Exploring network structure, dynamics, and function using NetworkX [R ] . 2008 .

PASZKE A , GROSS S , MASSA F , et al . PyTorch: an imperative style, high-performance deep learning library [M ] // Proceedings of the 33rd International Conference on Neural Information Processing Systems . 2019 .

WANG M , YU L , ZHENG D , et al . Deep graph library: towards efficient and scalable deep learning ongraphs [J ] . CoRR , 2019 , abs/1909. 01315 .

WU M , JIANG L , XIANG J , et al . One fuzzing strategy to rule them all [C ] // 44th IEEE/ACM 44th International Conference on Software Engineering(ICSE 2022) . 2022 : 1634 - 1645 .

GAN S , ZHANG C , CHEN P , et al . GREYONE: Data flow sensitive fuzzing [C ] // 29th USENIX Security Symposium (USENIX Security 20). 2020 : 2577 - 2594 .

CHEN P , CHEN H . Angora:efficient fuzzing by principled search [C ] // 2018 IEEE Symposium on Security and Privacy(SP) . 2018 : 711 - 725 .

STEPHENS N , GROSEN J , SALLS C , et al . Driller: augmenting fuzzing through selective symbolic execution .[C ] // NDSS , 2016 , 16 : 1 - 16 .

PENG H , SHOSHITAISHVILI Y , PAYER M . T-Fuzz: fuzzing by program transformation [C ] // 2018 IEEE Symposium on Security and Privacy (SP) . 2018 : 697 - 710 .

SHE D , PEI K , EPSTEIN D , et al . Neuzz: efficient fuzzing with neural program smoothing [C ] // 2019 IEEE Symposium on Security and Privacy (SP) . 2019 : 803 - 817 .

SHE D , KRISHNA R , YAN L , et al . MTFuzz: fuzzing with a multi-task neural network [C ] // Association for Computing Machinery . 2020 : 737 – 749 .

WU M , JIANG L , XIANG J , et al . Evaluating and improving neural program-smoothing-based fuzzing [C ] // Proceedings of the 44th International Conference on Software Engineering . 2022 : 847 - 858 .

RAJPAL M , BLUM W , SINGH R . Not all bytes are equal: Neural byte sieve for fuzzing [J ] . arXiv preprint arXiv: 1711.04596 , 2017 .

ZONG P , LV T , WANG D , et al . FuzzGuard: filtering out unreachable inputs in directed grey-box fuzzing through deep learning [C ] // 29th USENIX Security Symposium (USENIX Security 20). 2020 : 2255 - 2269 .

CHEN Y , AHMADI M , FARKHANI R M , et al . MEUZZ: Smart seed scheduling for hybrid fuzzing [C ] // 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020) . 2020 : 77 - 92 .

XU H , CHEN L , GAN S , et al . Graphuzz: data-driven seed scheduling for coverage-guided greybox fuzzing [J ] . ACM Trans Softw Eng Methodol , 2024 .

KONSTANTIN B , GODEFROID P , SINGH R . Deep reinforcement fuzzing [C ] // 2018 IEEE Security and Privacy Workshops (SPW) . 2018 : 116 - 122 .

BINOSI L , RULLO L , POLINO M , et al . RainFuzz: reinforcement-learning driven heat-maps for boosting coverage-guided fuzzing [C ] // Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods(ICPRAM 2023). 2023 : 39 - 50 .

肖天 , 江智昊 , 唐鹏 , 等 . 基于深度强化学习的高性能导向性模糊测试方案 [J ] . 网络与信息安全学报 , 2023 , 9 ( 2 ): 132 - 142 .

XIAO T , JIANG Z H , TANG P , et al . High-performance directional fuzzing scheme based on deep reinforcement learning [J ] . Chinese Journal of Network and Information Security , 2023 , 9 ( 2 ): 132 - 142 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Membership inference attack based on imitation learning for deep reinforcement learning

Adversarial method for malicious ELF file detection based on deep reinforcement learning

Survey of evolutionary kernel fuzzing

High-performance directional fuzzing scheme based on deep reinforcement learning

Research and implementation of fuzzing testing based on HTTP proxy

Related Author

ZHENG Haibin

QU Kangyun

CHEN Jinyin

FANG Zhiyang

WANG Junfeng

WU Di

GENG Jiaxuan

ZHANG Bocheng

Related Institution

Institute of Cyberspace Security, Zhejiang University of Technology

College of Information Engineering, Zhejiang University of Technology

Key Laboratory of Experimental Physics and Computational Mathematics

School of Cyber Science and Engineering, Sichuan University

College of Computer Science, Sichuan University

⁰