Survey of federated learning research

Chuanxin ZHOU; Yi SUN; Degang WANG; Huawei GE

doi:10.11959/j.issn.2096-109x.2021056

您当前的位置：

首页 >

文章列表页 >

Survey of federated learning research

TopicⅡ: Machine Learning and Security Application | 更新时间：2024-06-03

- Survey of federated learning research
- Chinese Journal of Network and Information Security Vol. 7, Issue 5, Pages: 77-92(2021)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2021056
  CLC： TP309.2
- Online First：2021-10，
  
  Published：15 October 2021
- 稿件说明：
移动端阅览
Chuanxin ZHOU, Yi SUN, Degang WANG, et al. Survey of federated learning research[J]. Chinese Journal of Network and Information Security, 2021, 7(5): 77-92.
DOI：

Chuanxin ZHOU, Yi SUN, Degang WANG, et al. Survey of federated learning research[J]. Chinese Journal of Network and Information Security, 2021, 7(5): 77-92. DOI： 10.11959/j.issn.2096-109x.2021056.

摘要

联邦学习由于能够在多方数据源聚合的场景下协同训练全局最优模型，近年来迅速成为安全机器学习领域的研究热点。首先，归纳了联邦学习定义、算法原理和分类；接着，深入分析了其面临的主要威胁与挑战；然后，重点对通信效率、隐私安全、信任与激励机制3个方向的典型研究方案对比分析，指出其优缺点；最后，结合边缘计算、区块链、5G等新兴技术对联邦学习的应用前景及研究热点进行展望。

Abstract

Federated learning has rapidly become a research hotspot in the field of security machine learning in recent years because it can train the global optimal model collaboratively without the need for multiple data source aggregation.Firstly

the federated learning framework

algorithm principle and classification were summarized.Then

the main threats and challenges it faced

were analysed indepth the comparative analysis of typical research programs in the three directions of communication efficiency

privacy and security

trust and incentive mechanism was focused on

and their advantages and disadvantages were pointed out.Finally

Combined with application of edge computing

blockchain

5G and other emerging technologies to federated learning

its future development prospects and research hotspots was prospected.

关键词

Keywords

references

微众银行AI项目组 . 联邦学习白皮书 V1.0 [R ] . 2018 .

WeBank AI Project Team . Federated learning white paper V1.0 [R ] . 2018 .

MCMAHAN H B , MOORE E , RAMAGE D , et al . Communication-efficient learning of deep networks from decentralized data [J ] . Artificial Intelligence and Statistics , 2017 : 1273 - 1282 .

杨强 . AI与数据隐私保护：联邦学习的破解之道 [J ] . 信息安全研究 , 2019 , 5 ( 11 ): 961 - 965 .

YANG Q . AI and data privacy protection:the cracking method of federated learning [J ] . Information Security Research . 2019 , 5 ( 11 ): 961 - 965 .

潘碧莹 , 丘海华 , 张家伦 . 不同数据分布的联邦机器学习技术研究 [M ] . 5G 网络创新研讨会 (2019) 论文集 , 2019 .

PAN B Y , QING H H , ZHANG J L . Research on federal machine learning technology with different data distribution [M ] . 5G Network Innovation Seminar (2019) Proceedings , 2019 .

YANG Q , LIU Y , CHEN T , et al . Federated machine learning:Concept and applications [J ] . ACM Transactions on Intelligent Systems and Technology (TIST) , 2019 , 10 ( 2 ): 1 - 19 .

HARD A , RAO K , MATHEWS R , et al . Federated learning for mobile keyboard prediction [J ] . arXiv preprint arXiv:1811.03604 , 2018 .

LI S , CHENG Y , LIU Y , et al . Abnormal client behavior detection in federated learning [J ] . arXiv preprint arXiv:1910.09933 , 2019 .

BONAWITZ K , IVANOV V , KREUTER B , et al . Practical secure aggregation for privacy-preserving machine learning [C ] // Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security . 2017 : 1175 - 1191 .

MANDAL K , GONG G , LIU C . NIKE-based fast privacy-preserving high-dimensional data aggregation for mobile devices [R ] . CACR Technical Report , 2018 .

CHENG K , FAN T , JIN Y , et al . Secureboost:a lossless federated learning framework [J ] . arXiv preprint arXiv:1901.08755 , 2019 .

KONEČNÝ J , MCMAHAN H B , YU F X , et al . Federated learning:strategies for improving communication efficiency [J ] . arXiv preprint arXiv:1610.05492 , 2016 .

WANG L P , WANG W , LI B . CMFL:mitigating communication overhead for federated learning [C ] // 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) . IEEE , 2019 : 954 - 964 .

CALDAS S , KONEČNY J , MC-MAHAN H B , et al . Expanding the reach of federated learning by reducing client resource requirements [J ] . arXiv preprint arXiv:1812.07210 , 2018 .

BHOWMICK A , DUCHI J , FREUDIGER J , et al . Protection against reconstruction and its applications in private federated learning [J ] . arXiv preprint arXiv:1812.00984 , 2018 .

CARLINI N , LIU C , KOS J , et al . The secret sharer:Measuring unintended neural network memorization ＆ extracting secrets [J ] . arXiv preprint arXiv:1802.08232 , 2018 .

FREDRIKSON M , LANTZ E , JHA S , et al . Privacy in pharmacogenetics:an end-to-end case study of personalized warfarin dosing [C ] // 23rd {USENIX} Security Symposium ({USENIX} Security 14) . 2014 : 17 - 32 .

MELIS L , SONG C , DE-CRISTOFARO E , et al . Exploiting unintended feature leakage in collaborative learning [C ] // 2019 IEEE Symposium on Security and Privacy (SP) . 2019 : 691 - 706 .

HITAJ B , ATENIESE G , PEREZ-CRUZ F , . Deep models under the GAN:information leakage from collaborative deep learning [C ] // Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security . 2017 : 603 - 618 .

GEI P J , BAUERMEISTER H , DRGE H , et al . Inverting gradients how easy is it to break privacy in federated learning [R ] . 2020 .

SONG M , WANG Z , ZHANG Z , et al . Analyzing user-level privacy attack against federated learning [J ] . IEEE Journal on Selected Areas in Communications , 2020 .

GEYER R C , KLEIN T , NABI M . Differentially private federated learning:a client level perspective [J ] . arXiv preprint arXiv:1712.07557 , 2017 .

MC MAHAN H B , RAMAGE D , TALWAR K , et al . Learning differentially private recurrent language models [J ] . arXiv preprint arXiv:1710.06963 , 2017 .

BONAWITZ K , IVANOV V , KREUTER B , et al . Practical secure aggregation for privacy-preserving machine learning [C ] // Proceedings of the 2017 ACM Sigsac Conference on Computer and Communications Security . 2017 : 1175 - 1191 .

MANDAL K , GONG G , LIU C . NIKE-based fast privacy-preserving high-dimensional data aggregation for mobile devices [R ] . CACR Technical Report , 2018 .

CHENG K , FAN T , JIN Y , et al . Secureboost:a lossless federated learning framework [J ] . arXiv preprint arXiv:1901.08755 , 2019 .

袁勇 , 王飞跃 . 区块链技术发展现状与展望 [J ] . 电子学报 , 2016 , 42 ( 4 ): 481 - 494 .

YUAN Y , WANG F Y . Blockchain:the state of the art and future trends [J ] . Acta Automatica Sinica , 2016 , 42 ( 4 ): 481 - 494 .

YANG R , YU F R , SI P , et al . Integrated blockchain and edge computing systems:a survey,some research issues and challenges [J ] . IEEE Communications Surveys ＆ Tutorials , 2019 , 21 ( 2 ): 1508 - 1532 .

DONG Y , CHEN X , SHEN L , et al . EaSTFLy:efficient and secure ternary federated learning [J ] . Computers ＆ Security , 2020 , 94 : 1 - 15 .

CAO H , LIU S , ZHAO R , et al . IFed:a novel federated learning framework for local differential privacy in power internet of things [J ] . International Journal of Distributed Sensor Networks , 2020 , 16 ( 5 ): 1 - 3 .

CHEN J , PAN X , MONGA R , et al . Revisiting distributed synchronous SGD [J ] . arXiv preprint arXiv:1604.00981 , 2016 .

SATTLER F , WIEDEMANN S , MÜLLER K R , , et al . Robust and communication-efficient federated learning from Non-IID data [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2019 .

XIAO P , CHENG S , STANKOVIC V , et al . Averaging is probably not the optimum way of aggregating parameters in federated learning [J ] . Entropy , 2020 , 22 ( 3 ): 314 .

LI T , SAHU A K , ZAHEER M , et al . Federated optimization in heterogeneous networks [J ] . arXiv preprint arXiv:1812.06127 , 2018 .

LIU W , CHEN L , CHEN Y , et al . Accelerating federated learning via momentum gradient descent [J ] . IEEE Transactions on Parallel and Distributed Systems , 2020 , 31 ( 8 ): 1754 - 1766 .

HUANG L , YIN Y , FU Z , et al . LoAdaBoost:loss-based AdaBoost federated machine learning with reduced computational complexity on IID and non-IID intensive care data [J ] . PLoS ONE , 2020 15 ( 4 ): 1 - 6 .

JIANG J , HU L , HU C , et al . BACombo—bandwidth-aware decentralized federated learning [J ] . Electronics , 2020 , 9 ( 3 ): 440 .

REISIZADEH A , MOKHTARI A , HASSANI H , et al . Fedpaq:a communication-efficient federated learning method with periodic averaging and quantization [C ] // International Conference on Artificial Intelligence and Statistics . 2020 : 2021 - 2031 .

LU X , LIAO Y , LIO P , et al . Privacy-preserving asynchronous federated learning mechanism for edge network computing [J ] . IEEE Access , 2020 , 8 : 48970 - 48981 .

HE L , BIAN A , JAGGI M . Cola:decentralized linear learning [C ] // Advances in Neural Information Processing Systems . 2018 : 4536 - 4546 .

LALITHA A , WANG X , KILINC O , et al . Decentralized Bayesian learning over graphs [J ] . arXiv preprint arXiv:1905.10466 , 2019 .

LIN T , STICH S U , PATEL K K , et al . Don't use large mini-batches,use local SGD [J ] . arXiv preprint arXiv:1808.07217 , 2018 .

LIU L , ZHANG J , SONG S H , et al . Edge-assisted hierarchical federated learning with non-iid data [J ] . arXiv preprint arXiv:1905.06641 , 2019 .

SHARMA P K , PARK J H , CHO K . Blockchain and federated learning-based distributed computing defence framework for sustainable society [J ] . Sustainable Cities and Society , 2020 :102220.

ZHANG J , ZHAO Y , WANG J , et al . FedMEC:improving efficiency of differentially private federated learning via mobile edge computing [J ] . Mobile Networks and Applications , 2020 : 1 - 13 .

刘俊旭 , 孟小峰 . 机器学习的隐私保护研究综述 [J ] . 计算机研究与发展 , 2020 , 57 ( 2 ): 346 .

LIU J X , MENG X F . A survey of research on privacy protection in machine learning [J ] . Computer Research and Development , 2020 , 57 ( 2 ): 346 .

DWORK C , MC-SHERRY F , NISSIM K , et al . Calibrating noise to sensitivity in private data analysis [C ] // Theory of Cryptography Conference . 2006 : 265 - 284 .

叶青青 , 孟小峰 , 朱敏杰 , 等 . 本地化差分隐私研究综述 [J ] . 软件学报 , 2018 , 29 ( 7 ): 159 - 183 .

YE Q Q , MENG X F , ZHU M J , et al . Survey of localized differential privacy research [J ] . Journal of Software , 2018 , 29 ( 7 ): 159 - 183 .

苏冠通 , 徐茂桐 . 安全多方计算技术与应用综述 [J ] . 信息通信技术与政策 , 2019 ( 5 ): 19 - 22 .

SU G T , XU M T . Survey of secure multiparty computing technology and application [J ] . Information and Communication Technol ogy and Policy , 2019 ( 5 ): 19 - 22 .

DOLEV D , YAO A . On the security of public key protocols [J ] . IEEE Transactions on Information Theory , 1983 , 29 ( 2 ): 198 - 208 .

RIVEST R L , SHAMIR A , ADLEMAN L . A method for obtaining digital signatures and public-key cryptosystems [J ] . Communications of the ACM , 1978 , 21 ( 2 ): 120 - 126 .

THAKKAR O , ANDREW G , MC-MAHAN H B , . Differentially private learning with adaptive clipping [J ] . arXiv preprint arXiv:1905.03871 , 2019 .

HAO M , LI H , LUO X , et al . Efficient and privacy-enhanced federated learning for industrial artificial intelligence [J ] . IEEE Transactions on Industrial Informatics , 2019 , 16 ( 10 ): 6532 - 6542 .

AONO Y , HAYASHI T , WANG L , et al . Privacy-preserving deep learning via additively homomorphic encryption [J ] . IEEE Transactions on Information Forensics and Security , 2017 , 13 ( 5 ): 1333 - 1345 .

LIU X , LI H , XU G , et al . Adaptive privacy-preserving federated learning [J ] . Peer to Peer Networking and Applications , 2020 , 13 : 2356 - 2366 .

HUANG X , DING Y , JIANG Z L , et al . DP-FL:a novel differentially private federated learning framework for the unbalanced data [J ] . World Wide Web , 2020 : 1 - 17 .

WEI K , LI J , DING M , et al . Federated learning with differential privacy:algorithms and performance analysis [J ] . IEEE Transactions on Information Forensics and Security , 2020 , 15 : 3454 - 3469 .

XU G , LI H , LIU S , et al . VerifyNet:secure and verifiable federated learning [J ] . IEEE Transactions on Information Forensics and Security , 2019 , 15 : 911 - 926 .

LU Y , HUANG X , DAI Y , et al . Differentially private asynchronous federated learning for mobile edge computing in urban informatics [J ] . IEEE Transactions on Industrial Informatics , 2019 .

LU Y , HUANG X , DAI Y , et al . Federated learning for data privacy preservation in vehicular cyber-physical systems [J ] . IEEE Network , 2020 , 34 ( 3 ): 50 - 56 .

HU R , GUO Y , LI H , et al . Personalized federated learning with differential privacy [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 10 ): 9530 - 9539 .

FANG M , CAO X , JIA J , et al . Local model poisoning attacks to Byzantine-robust federated learning [J ] . arXiv preprint arXiv:1911.11815 , 2019 .

ZHAO Y , CHEN J , ZHANG J , et al . Detecting and mitigating poisoning attacks in federated learning using generative adversarial networks [J ] . Concurrency and Computation:Practice and Experience , 2020 : 1 - 2 .

KANG J , XIONG Z , NIYATO D , et al . Incentive mechanism for reliable federated learning:a joint optimization approach to combining reputation and contract theory [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 6 ): 10700 - 10714 .

FUNG C , YOON C J M , BESCHASTNIKH I . Mitigating sybils in federated learning poisoning [J ] . arXiv preprint arXiv:1808.04866 , 2018 .

CHEN Y , LUO F , LI T , et al . A training-integrity privacy-preserving federated learning scheme with trusted execution environment [J ] . Information Sciences , 2020 , 522 : 69 - 79 .

LYU L , YU J , NANDAKUMAR K , et al . Towards fair and privacy-preserving federated deep models [J ] . IEEE Transactions on Parallel and Distributed Systems , 2020 , 31 ( 11 ): 2524 - 2541 .

MAJEED U , HONG C S . FLchain:federated learning via MEC-enabled blockchain network [C ] // 2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS) . 2019 : 1 - 4 .

ARACHCHIGE P C M , BERTOK P , KHALIL I , et al . A trustworthy privacy preserving framework for machine learning in industrial iot systems [J ] . IEEE Transactions on Industrial Informatics , 2020 , 16 ( 9 ): 6092 - 6102 .

LU Y , HUANG X , ZHANG K , et al . Blockchain empowered asynchronous federated learning for secure data sharing in internet of vehicles [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 4 ): 4298 - 4311 .

POKHREL S R . Towards efficient and reliable federated learning using blockchain for autonomous vehicles [J ] . Computer Networks , 2020 :107431.

KIM H , PARK J , BENNIS M , et al . On-device federated learning via blockchain and its latency analysis [J ] . arXiv preprint arXiv:1808.03949 , 2018 .

WENG J , WENG J , ZHANG J , et al . Deepchain:auditable and privacy-preserving deep learning with blockchain-based incentive [J ] . IEEE Transactions on Dependable and Secure Computing , 2019 .

KIM Y J , HONG C S . Blockchain-based node-aware dynamic weighting methods for improving federated learning performance [C ] // 20th Asia-Pacific Network Operations and Management Symposium (APNOMS) . 2019 : 1 - 4 .

ZHAN Y , LI P , QU Z , et al . A learning-based incentive mechanism for federated learning [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 7 ): 6360 - 6368 .

PREUVENEERS D , RIMMER V , TSINGENOPOULOS I , et al . Chained anomaly detection models for federated learning:an intrusion detection case study [J ] . Applied Sciences , 2018 , 8 ( 12 ): 2663 .

ZHU X , LI H , YU Y . Blockchain-based privacy preserving deep learning [C ] // International Conference on Information Security and Cryptology . 2018 : 370 - 383 .

QU Y , GAO L , LUAN T H , et al . Decentralized privacy using blockchain-enabled federated learning in fog computing [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 6 ): 5171 - 5183 .

LIU Y , PENG J , KANG J , et al . A secure federated learning framework for 5G networks [J ] . arXiv preprint arXiv:2005.05752 , 2020 .

KAIROUZ P , MC-MAHAN H B , AVENT B , et al . Advances and open problems in federated learning [J ] . arXiv preprint arXiv:1912.04977 , 2019 .

LI T , SAHU A K , TALWALKAR A , et al . Federated learning:challenges,methods,and future directions [J ] . arXiv preprint arXiv:1908.07873 , 2019 .

SMITH V , CHIANG C K , SANJABI M , et al . Federated multi-task learning [C ] // Advances in Neural Information Processing Systems . 2017 : 4424 - 4434 .

ANG F , CHEN L , ZHAO N , et al . Robust federated learning with noisy communication [J ] . IEEE Transactions on Communications , 2020 .

NIKNAM S , DHILLON H S , REED J H . Federated learning for wireless communications:motivation,opportunities,and challenges [J ] . IEEE Communications Magazine , 2020 , 58 ( 6 ): 46 - 51 .

REN J , WANG H , HOU T , et al . Federated learning-based computation offloading optimization in edge computing-supported internet of things [J ] . IEEE Access , 2019 , 7 : 69194 - 69201 .

WANG X , HAN Y , WANG C , et al . In-edge AI:intelligentizing mobile edge computing,caching and communication by federated learning [J ] . IEEE Network , 2019 , 33 ( 5 ): 156 - 165 .

WANG S , TUOR T , SALONIDIS T , et al . Adaptive federated learning in resource constrained edge computing systems [J ] . IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 6 ): 1205 - 1221 .

Views

7703

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Double layer federated security learning architecture for artificial intelligence of things

FL-IoV: comprehensive intrusion detection method for internet of vehicles based on federated learning

Blockchain based adaptive federated learning in computational power network

Blockchain-based cross-data center anonymous and verifiable identity authentication scheme

Survey on blockchain privacy protection techniques in cryptography

Related Author

ZHENG Chengbo

YAN Haonan

FU Caili

ZHANG Dong

LI Hui

WANG Bin

JIANG Yuchang

JIANG Libing

Related Institution

School of Cyber Engineering, Xidian University

Hangzhou Hikvision Digital Technology Co.,Ltd

National Computer Network Emergency Response Technical Team/Coordination Center of China(CNCERT/CC)

Zhejiang Key Laboratory of Artificial Intelligence of Things（AIoT）Network and Data Security

Beijing Unionpay Card Technology Co., Ltd., Shenzhen Branch

⁰