Malicious code within model detection method based on model similarity

Degang WANG; Yi SUN; Chuanxin ZHOU; Qi GAO; Fan YANG

doi:10.11959/j.issn.2096-109x.2023056

您当前的位置：

首页 >

文章列表页 >

Malicious code within model detection method based on model similarity

Papers | 更新时间：2024-06-03

- Malicious code within model detection method based on model similarity
- Chinese Journal of Network and Information Security Vol. 9, Issue 4, Pages: 90-103(2023)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2023056
  CLC： TP309.2
- Online First：2023-08，
  
  Published：25 August 2023
- 稿件说明：
移动端阅览
Degang WANG, Yi SUN, Chuanxin ZHOU, et al. Malicious code within model detection method based on model similarity[J]. Chinese Journal of Network and Information Security, 2023, 9(4): 90-103.
DOI：

Degang WANG, Yi SUN, Chuanxin ZHOU, et al. Malicious code within model detection method based on model similarity[J]. Chinese Journal of Network and Information Security, 2023, 9(4): 90-103. DOI： 10.11959/j.issn.2096-109x.2023056.

摘要

联邦学习主要通过源数据不出本地而仅交互模型参数的方式保护参与共享用户数据的隐私安全，然而其仍然面临众多安全挑战，目前研究者主要针对如何增强模型隐私保护和检测恶意模型攻击等问题展开较为广泛的研究，然而利用联邦学习过程中频繁交互的模型数据进行恶意代码夹带导致风险扩散的问题鲜有研究。针对联邦学习训练过程中通过模型传递恶意代码导致风险扩散的问题，提出一种基于模型相似度的模型恶意代码夹带检测方法。通过分析联邦学习本地模型与全局模型的迭代过程，提出计算模型距离的方法，并使用模型距离量化模型之间的相似度，最终根据各客户端模型之间的相似度对携带恶意代码的模型进行检测。实验结果表明，提出的检测方法具有较好的性能指标，当训练集为独立同分布时，在178 MB大小的模型中嵌入0.375 MB恶意代码，检测方法的真正率为82.9%，误报率为1.8%；嵌入0.75 MB恶意代码时，检测方法的真正率为 96.6%，误报率为 0.38%。当训练集为非独立同分布时，检测方法的准确率随恶意代码嵌入率以及联邦学习训练轮数的增加而增加。在对恶意代码进行加密的情况下，提出的检测方法仍然能够达到 90%以上的准确率。在多攻击者的场景中，攻击者数量已知与未知时的检测方法准确率均能保持在90%左右。

Abstract

The privacy of user data in federated learning is mainly protected by exchanging model parameters instead of source data.However

federated learning still encounters many security challenges.Extensive research has been conducted to enhance model privacy and detect malicious model attacks.Nevertheless

the issue of risk-spreading through malicious code propagation during the frequent exchange of model data in the federated learning process has received limited attention.To address this issue

a method for detecting malicious code within models

based on model similarity

was proposed.By analyzing the iterative process of local and global models in federated learning

a model distance calculation method was introduced to quantify the similarity between models.Subsequently

the presence of a model carrying malicious code is detected based on the similarity between client models.Experimental results demonstrate the effectiveness of the proposed detection method.For a 178MB model containing 0.375MB embedded malicious code in a training set that is independent and identically distributed

the detection method achieves a true rate of 82.9% and a false positive rate of 1.8%.With 0.75MB of malicious code embedded in the model

the detection method achieves a true rate of 96.6% and a false positive rate of 0.38%.In the case of a non-independent and non-identically distributed training set

the accuracy of the detection method improves as the rate of malicious code embedding and the number of federated learning training rounds increase.Even when the malicious code is encrypted

the accuracy of the proposed detection method still achieves over 90%.In a multi-attacker scenario

the detection method maintains an accuracy of approximately 90% regardless of whether the number of attackers is known or unknown.

关键词

Keywords

references

MCMAHAN B , MOORE E , RAMAGE D , et al . Communication-efficient learning of deep networks from decentralized data [C ] // Artificial Intelligence and Statistics (AISTATS) . 2017 : 1273 - 1282 .

FREDRIKSON M , JHA S , RISTENPART T . Model inversion attacks that exploit confidence information and basic countermeasures [C ] // Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS) . 2015 : 1322 - 1333 .

HITAJ B , ATENIESE G , PEREZ-CRUZ F . Deep models under the GAN:information leakage from collaborative deep learning [C ] // Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS) . 2017 : 603 - 618 .

SHOKRI R , STRONATI M , SONG C , et al . Membership inference attacks against machine learning models [C ] // 2017 IEEE Symposium on Security and Privacy (S＆P) . 2017 : 3 - 18 .

TRAMÈR F , ZHANG F , JUELS A , et al . Stealing machine learning models via prediction {APIs} [C ] // 25th USENIX Security Symposium (USENIX Security 16) . 2016 : 601 - 618 .

GU T , DOLAN-GAVITT B , GARG S . Badnets:identifying vulnerabilities in the machine learning model supply chain [J ] . arXiv Preprint arXiv:1708.06733 , 2017 .

JI Y , ZHANG X , WANG T . Backdoor attacks against learning systems [C ] // 2017 IEEE Conference on Communications and Network Security . 2017 : 1 - 9 .

BAGDASARYAN E , VEIT A , HUA Y , et al . How to backdoor federated learning [C ] // International Conference on Artificial Intelligence and Statistics (AISTATS 2020) . 2020 : 2938 - 2948 .

JI Y , ZHANG X , JI S , et al . Model-reuse attacks on deep learning systems [C ] // Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (CCS 2018) . 2018 : 349 - 363 .

YAO Y , LI H , ZHENG H , et al . Regula sub-rosa:latent backdoor attacks on deep neural networks [J ] . arXiv Preprint arXiv:1905.10447 , 2019 .

YAO Y , LI H , ZHENG H , et al . Latent backdoor attacks on deep neural networks [C ] // Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS 2019) . 2019 : 2041 - 2055 .

BHAGOJI A N , CHAKRABORTY S , MITTAL P , et al . Analyzing federated learning through an adversarial lens [C ] // International Conference on Machine Learning (ICML) . 2019 : 634 - 643 .

CARLINI N , LIU C , ERLINGSSON Ú , et al . The secret sharer:evaluating and testing unintended memorization in neural networks [C ] // 28th USENIX Security Symposium (USENIX Security 19) . 2019 : 267 - 284 .

LIU T , LIU Z , LIU Q , et al . StegoNet:turn deep neural network into a stegomalware [C ] // Annual Computer Security Applications Conference (ACSAC'20) . 2020 : 928 - 938 .

WANG Z , LIU C , CUI X . EvilModel:hiding malware inside of neural network models [C ] // 2021 IEEE Symposium on Computers and Communications (ISCC) . 2021 : 1 - 7 .

WANG D G , SUN Y , ZHOU C X . A covert communication method based on gradient model [C ] // 2021 IEEE 6th International Conference on Signal and Image Processing (ICSIP) . 2021 : 926 - 930 .

周传鑫 , 孙奕 , 汪德刚 , 等 . 联邦学习研究综述 [J ] . 网络与信息安全学报 , 2021 , 7 ( 5 ): 77 - 92 .

ZHOU C X , SUN Y , WANG D G , et al . Survey of federated learning research [J ] . Chinese Journal of Network and Information Security , 2021 , 7 ( 5 ): 77 - 92 .

LYU L , YU H , ZHAO J , et al . Threats to federated learning [M ] // Federated Learning . Berlin : Springer , 2020 : 3 - 16 .

PROVOS N , HONEYMAN P . Hide and seek:an introduction to steganography [J ] . IEEE Security ＆ Privacy , 2003 , 1 ( 3 ): 32 - 44 .

CHAN C K , CHENG L M . Hiding data in images by simple LSB substitution [J ] . Pattern Recognition , 2004 , 37 ( 3 ): 469 - 474 .

KAHAN W . IEEE standard 754 for binary floating-point arithmetic [J ] . Lecture Notes on the Status of IEEE , 1996 :11.

FRIDRICH J , GOLJAN M , DU R . Reliable detection of LSB steganography in color and grayscale images [C ] // Proceedings of the 2001 Workshop on Multimedia and Security:New Challenges (MM＆Sec) . 2001 : 27 - 30 .

InQuest malware-samples [EB ] . 2022 .

范铭 , 刘烃 , 刘均 , 等 . 安卓恶意软件检测方法综述 [J ] . 中国科学：信息科学 , 2020 , 50 ( 8 ): 1148 - 1177 .

FAN M , LIU T , LIU J , et al . Android malware detection:a survey [J ] . Scientia Sinica Informationis , 2020 , 50 ( 8 ): 1148 - 1177 .

Views

388

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

DP-FedDRC: differentially private personalized federated learning to alleviate dimension collapse

Anti-attack federated framework based on dynamic clustering and reputation evaluation

Survey on malicious code dynamic analysis evasion and anti-evasion techniques

Financial fraud detection model based on risk weight aggregation federated learning

FL-IoV: comprehensive intrusion detection method for internet of vehicles based on federated learning

Related Author

Hu Changhui

Zhang Yuhao

Wang Minli

ZHANG Lin

LUO Qirui

ZHANG Weiyi

WANG Chenyang

PENG Guojun

Related Institution

School of Cyberspace Security (School of Cryptography), Hainan University

College of Computer, Nanjing University of Posts and Telecommunications

State Key Laboratory of Public Big Data, Guizhou University

School of Cyber Science and Engineering, Wuhan University

Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education

⁰