Privacy-preserving federated learning framework with irregular-majority users

Qianxin CHEN; Renwan BI; Jie LIN; Biao JIN; Jinbo XIONG

doi:10.11959/j.issn.2096-109x.2022011

您当前的位置：

首页 >

文章列表页 >

Privacy-preserving federated learning framework with irregular-majority users

Research and Development | 更新时间：2024-06-03

- Privacy-preserving federated learning framework with irregular-majority users
- Chinese Journal of Network and Information Security Vol. 8, Issue 1, Pages: 139-150(2022)
- 作者机构：
  
  1. 福建师范大学计算机与网络空间安全学院，福建福州350117
  2. 福建师范大学福建省网络安全与密码技术重点实验室，福建福州 350007
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2022011
  CLC： TP309
- Online First：2022-02，
  
  Published：15 February 2022
- 稿件说明：
移动端阅览
Qianxin CHEN, Renwan BI, Jie LIN, et al. Privacy-preserving federated learning framework with irregular-majority users[J]. Chinese Journal of Network and Information Security, 2022, 8(1): 139-150.
DOI：

Qianxin CHEN, Renwan BI, Jie LIN, et al. Privacy-preserving federated learning framework with irregular-majority users[J]. Chinese Journal of Network and Information Security, 2022, 8(1): 139-150. DOI： 10.11959/j.issn.2096-109x.2022011.

摘要

针对联邦学习存在处理大多数不规则用户易引起聚合效率降低，以及采用明文通信导致参数隐私泄露的问题，基于设计的安全除法协议构建针对不规则用户鲁棒的隐私保护联邦学习框架。该框架通过将模型相关计算外包给两台边缘服务器以减小采用同态加密产生的高额计算开销，不仅允许模型及其相关信息以密文形式在边缘服务器上进行密文聚合，还支持用户在本地进行模型可靠性计算以减小传统方法采用安全乘法协议造成的额外通信开销。在该框架的基础上，为更精准评估模型的泛化性能，用户完成本地模型参数更新后，利用边缘服务器下发的验证集与本地持有的验证集联合计算模型损失值，并结合损失值历史信息动态更新模型可靠性以作为模型权重。进一步，在模型可靠性先验知识指导下进行模型权重缩放，将密文模型与密文权重信息交由边缘服务器对全局模型参数进行聚合更新，保证全局模型变化主要由高质量数据用户贡献，提高收敛速度。通过HybridArgument模型进行安全性分析，论证表明PPRFL（privacy-preserving robust federated learning）可以有效保护模型参数以及包括用户可靠性等中间交互参数的隐私。实验结果表明，当联邦聚合任务中的所有参与方均为不规则用户时，PPRFL 方案准确率仍然能达到 92%，收敛效率较 PPFDL （privacy-preserving federated deep learning with irregular users）提高1.4倍；当联邦聚合任务中80%用户持有的训练数据都为噪声数据时，PPRFL方案准确率仍然能达到89%，收敛效率较PPFDL提高2.3倍。

Abstract

In response to the existing problems that the federated learning might lead to the reduction of aggregation efficiency by handing the majority of irregular users and the leak of parameter privacy by adopting plaintext communication

a framework of privacy-preserving robust federated learning was proposed for ensuring the robustness of the irregular user based on the designed security division protocol.PPRFL could enable the model and its related information to aggregate in ciphertext on the edge server facilitate users to calculate the model reliability locally for reducing the additional communication overhead caused by the adoption of the security multiplication protocol in conventional methods

apart from lowering the high computational overhead resulted from homomorphic encryption with outsourcing computing to two edge servers.Based on this

user could calculate the loss value of the model through jointly using the verification sets issued by the edge server and that held locally after parameter updating of the local model.Then the model reliability could be dynamically updated as the model weight together with the historic information of the loss value.Further

the model weight was scaled under the guidance of prior knowledge

and the ciphertext model and ciphertext weight information are sent to the edge server to aggregate and update the global model parameters

ensuring that global model changes are contributed by high-quality data users

and improving the convergence speed.Through the security analysis of the Hybrid Argument model

the demonstration shows that PPRFL can effectively protect the privacy of model parameters and intermediate interaction parameters including user reliability.The experimental results show that the PPRFL scheme could still achieve the accuracy of 92% when all the participants in the federated aggregation task are irregular users

with the convergence efficiency 1.4 times higher than that of the PPFDL.Besides

the PPRFL scheme could still reach the accuracy of 89% when training data possessed by 80% of the users in the federated aggregation task were noise data

with the convergence efficiency 2.3 times higher than that of the PPFDL.

关键词

Keywords

references

LI T , SAHU A K , TALWALKAR A , et al . Federated learning:challenges,methods,and future directions [J ] . IEEE Signal Processing Magazine , 2020 , 37 ( 3 ): 50 - 60 .

YANG Q , LIU Y , CHEN T J , et al . Federated machine learning [J ] . ACM Transactions on Intelligent Systems and Technology , 2019 , 10 ( 2 ): 1 - 19 .

梁应敞 , 谭俊杰 , 智能无线通信技术研究概况 [J ] . 通信学报 , 2020 , 41 ( 7 ): 1 - 17 .

LIANG Y C , TAN J J , NIYATO D . Overview on intelligent wireless communication technology [J ] . Journal on Communications , 2020 , 41 ( 7 ): 1 - 17 .

杨强 . AI与数据隐私保护：联邦学习的破解之道 [J ] . 信息安全研究 , 2019 , 5 ( 11 ): 961 - 965 .

YANG Q . AI and data privacy protection:the way to federated learning [J ] . Journal of Information Security Research , 2019 , 5 ( 11 ): 961 - 965 .

谭清尹 , 曾颖明 , 韩叶 , 等 . 神经网络后门攻击研究 [J ] . 网络与信息安全学报 , 2021 , 7 ( 3 ): 46 - 58 .

TAN Q Y , ZENG Y M , HAN Y , et al . Survey on backdoor attacks targeted on neural network [J ] . Chinese Journal of Network and Information Security , 2021 , 7 ( 3 ): 46 - 58 .

MELIS L , SONG C Z , DE CRISTOFARO E , et al . Exploiting unintended feature leakage in collaborative learning [C ] // Proceedings of 2019 IEEE Symposium on Security and Privacy . Piscataway:IEEE Press , 2019 : 691 - 706 .

周传鑫 , 孙奕 , 汪德刚 , 等 . 联邦学习研究综述 [J ] . 网络与信息安全学报 , 2021 , 7 ( 5 ): 77 - 92 .

ZHOU C X , SUN Y , WANG D G , et al . Survey of federated learning research [J ] . Chinese Journal of Network and Information Security , 2021 , 7 ( 5 ): 77 - 92 .

HITAJ B , ATENIESE G , PEREZ-CRUZ F , . Deep models under the GAN:information leakage from collaborative deep learning [C ] // Proceedings of CCS '17:Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security . 2017 : 603 - 618 .

MOTHUKURI V , PARIZI R M , POURIYEH S , et al . A survey on security and privacy of federated learning [J ] . Future Generation Computer Systems , 2021 , 115 : 619 - 640 .

WAGH S , GUPTA D , CHANDRAN N . SecureNN:3-party secure computation for neural network training [J ] . Proceedings on Privacy Enhancing Technologies , 2019 , 2019 ( 3 ): 26 - 49 .

XU R H , BARACALDO N , ZHOU Y , et al . HybridAlpha:an efficient approach for privacy-preserving federated learning [C ] // Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security . 2019 : 13 - 23 .

方晨 , 郭渊博 , 王一丰 , 等 . 基于区块链和联邦学习的边缘计算隐私保护方法 [J ] . 通信学报 , 2021 , 42 ( 11 ): 28 - 40 .

FANG C , GUO Y B , WANG Y F , et al . Edge computing privacy protection method based on blockchain and federated learning [J ] . Journal on Communications , 2021 , 42 ( 11 ): 28 - 40 .

KANAGAVELU R , LI Z X , SAMSUDIN J , et al . Two-phase multi-party computation enabled privacy-preserving federated learning [C ] // Proceedings of 2020 20th IEEE/ACM International Symposium on Cluster,Cloud and Internet Computing (CCGRID) . 2020 : 410 - 419 .

董业 , 侯炜 , 陈小军 , 等 . 基于秘密分享和梯度选择的高效安全联邦学习 [J ] . 计算机研究与发展 , 2020 , 57 ( 10 ): 2241 - 2250 .

DONG Y , HOU W , CHEN X J , et al . Efficient and secure federated learning based on secret sharing and gradients selection [J ] . Journal of Computer Research and Development , 2020 , 57 ( 10 ): 2241 - 2250 .

PHONG L T , AONO Y , HAYASHI T , et al . Privacy-preserving deep learning via additively homomorphic encryption [J ] . IEEE Transactions on Information Forensics and Security , 2018 , 13 ( 5 ): 1333 - 1345 .

张泽辉 , 富瑶 , 高铁杠 . 支持数据隐私保护的联邦深度神经网络模型研究 [J ] . 自动化学报 , 2020 .

ZHANG Z H , FU Y , GAO T G . Research on federated deep neural network model for data privacy protection [J ] . Acta Automatica Sinica , 2020 .

ZHAO L C , WANG Q , ZOU Q , et al . Privacy-preserving collaborative deep learning with unreliable participants [J ] . IEEE Transactions on Information Forensics and Security , 2020 , 15 : 1486 - 1500 .

JAYARAMAN B , EVANS D . Evaluating differentially private machine learning in practice [C ] // Proceedings of the28th USENIX Security Symposium . 2019 : 1895 - 1912 .

成艺 . 联合学习环境下保护隐私的数据聚合技术研究 [D ] . 成都:电子科技大学 , 2020 : 17 - 45 .

CHENG Y . Research on data aggregation technology based on privacy-preserving in federated learning [D ] . Chengdu:University of Electronic Science and Technology of China , 2020 : 17 - 45 .

SHAMIR A . How to share a secret [J ] . Communications of the ACM , 1979 , 22 ( 11 ): 612 - 613 .

XU G W , LI H W , ZHANG Y , et al . Privacy-preserving federated deep learning with irregular users [J ] . IEEE Transactions on Dependable and Secure Computing , 2020 ,(99):1.

ZHENG Y F , DUAN H Y , WANG C . Learning the truth privately and confidently:encrypted confidence-aware truth discovery in mobile crowdsensing [J ] . IEEE Transactions on Information Forensics and Security , 2018 , 13 ( 10 ): 2475 - 2489 .

TIAN Y L , LI T , XIONG J B , et al . A blockchain-based machine learning framework for edge services in IoT [J ] . IEEE Transactions on Industrial Informatics , 2022 , 18 ( 3 ): 1918 - 1929 .

XIONG J B , BI R W , ZHAO M F , et al . Edge-assisted privacy-preserving raw data sharing framework for connected autonomous vehicles [J ] . IEEE Wireless Communications , 2020 , 27 ( 3 ): 24 - 30 .

BONAWITZ K , EICHNER H , GRIESKAMP W , et al . Towards federated learning at scale:system design [J ] . arXiv preprint arXiv:1902.01046 , 2019 .

MCMAHAN H B , MOORE E , RAMAGE D , et al . Communication-efficient learning of deep networks from decentralized data [C ] // Proceedings of the 20th International Conference on Artificial Intelligence and Statistics . 2017 : 1273 - 1282 .

PAILLIER P , . Public-key cryptosystems based on composite degree residuosity classes [C ] // Proceedings of the International Conference on the Theory and Applications of Cryptographic Techniques . 1999 : 223 - 238 .

ACAR A , AKSU H , ULUAGAC A S , et al . A survey on homomorphic encryption schemes [J ] . ACM Computing Surveys , 2019 , 51 ( 4 ): 1 - 35 .

CANETTI R , FEIGE U , GOLDREICH O , et al . Adaptively secure multi-party computation [C ] // Proceedings of the twenty-eighth annual ACM Symposium on Theory of Computing . 1996 : 639 - 648 .

HENDERSON M , THOMSON B , WILLIAMS J D . The third dialog state tracking challenge [C ] // Proceedings of 2014 IEEE Spoken Language Technology Workshop . 2014 : 324 - 329 .

BUDZIANOWSKI P , WEN T H , TSENG B H , et al . MultiWOZ A large-scale multi-domain wizard-of-o z dataset for task-oriented dialogue modelling [C ] // Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . 2018 : 5016 - 5026 .

SHOKRI R , SHMATIKOV V . Privacy-preserving deep learning [C ] // Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security . 2015 : 1310 - 1321 .

KANG J W , XIONG Z H , NIYATO D , et al . Toward secure blockchain-enabled Internet of vehicles:optimizing consensus management using reputation and contract theory [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 3 ): 2906 - 2920 .

KANG J W , XIONG Z H , NIYATO D , et al . Incentive mechanism for reliable federated learning:a joint optimization approach to combining reputation and contract theory [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 6 ): 10700 - 10714 .

KANG J W , XIONG Z H , NIYATO D , et al . Reliable federated learning for mobile networks [J ] . IEEE Wireless Communications , 2020 , 27 ( 2 ): 72 - 80 .

CATALANO D , FIORE D . Using linearly-homomorphic encryption to evaluate degree-2 functions on encrypted data [C ] // Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security . 2015 : 1518 - 1529 .

LECUN Y , BOTTOU L , BENGIO Y , et al . Gradient-based learning applied to document recognition [J ] . Proceedings of the IEEE , 1998 , 86 ( 11 ): 2278 - 2324 .

Views

684

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

DP-FedDRC: differentially private personalized federated learning to alleviate dimension collapse

Anti-attack federated framework based on dynamic clustering and reputation evaluation

Financial fraud detection model based on risk weight aggregation federated learning

FL-IoV: comprehensive intrusion detection method for internet of vehicles based on federated learning

FLWD:A Webshell detection method based on federated learning

Related Author

Hu Changhui

Zhang Yuhao

Wang Minli

ZHANG Lin

LUO Qirui

ZHANG Weiyi

ZHAO Qinxin

LIU Juan

Related Institution

School of Cyberspace Security (School of Cryptography), Hainan University

College of Computer, Nanjing University of Posts and Telecommunications

School of Software, Nanjing University

School of Cyber Space Security, Beijing Jiaotong University

Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University

⁰