Privacy-enhanced federated learning scheme based on generative adversarial networks

Feng YU; Qingxin LIN; Hui LIN; Xiaoding WANG

doi:10.11959/j.issn.2096-109x.2023043

您当前的位置：

首页 >

文章列表页 >

Privacy-enhanced federated learning scheme based on generative adversarial networks

Papers | 更新时间：2024-06-03

- Privacy-enhanced federated learning scheme based on generative adversarial networks
- Chinese Journal of Network and Information Security Vol. 9, Issue 3, Pages: 113-122(2023)
- 作者机构：
  
  1. 福建师范大学计算机与网络空间安全学院，福建福州 350017
  2. 网络安全与教育信息化福建省高校工程研究中心（福建师范大学），福建福州 350017
  3. 福州大学至诚学院，福建福州 350002
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2023043
  CLC： TP393
- Online First：2023-06，
  
  Published：25 June 2023
- 稿件说明：
移动端阅览
Feng YU, Qingxin LIN, Hui LIN, et al. Privacy-enhanced federated learning scheme based on generative adversarial networks[J]. Chinese Journal of Network and Information Security, 2023, 9(3): 113-122.
DOI：

Feng YU, Qingxin LIN, Hui LIN, et al. Privacy-enhanced federated learning scheme based on generative adversarial networks[J]. Chinese Journal of Network and Information Security, 2023, 9(3): 113-122. DOI： 10.11959/j.issn.2096-109x.2023043.

摘要

联邦学习作为一种分布式机器学习范式，其具有隐私保护能力和异构协作等特性，引起了研究者极大的关注。然而，研究工作表明通过梯度可以确定一个确切的数据记录或一个具有特定属性的数据记录是否包含在其他参与者的批处理中，甚至揭露参与者的训练数据，通常称之为“梯度泄露”。同时，当前隐私增强联邦学习方法的工作可能存在准确率下降或者计算通信开销增加等问题，甚至引发新的不安全因素。因此，提出一种差分隐私增强的生成对抗网络模型，该模型向vanilla GAN中引入了识别器，通过生成器与鉴别器、生成器与识别器两个博弈过程，生成器合成的数据尽可能接近输入数据的同时满足差分隐私的约束。将此模型应用到联邦学习框架中，在一定程度上保证了模型准确率，并且提高了联邦学习框架的隐私保护能力。仿真实验验证了所提方案在客户端/服务器联邦学习架构下的有效性，相比 DP-SGD 方法，所提方案平衡了数据隐私性与实用性而不是以牺牲准确率为代价来增强隐私保护能力。从理论上分析了所提模型在点对点（P2P，peer-to-peer）架构下的可用性，并讨论了未来研究工作。

Abstract

Federated learning

a distributed machine learning paradigm

has gained a lot of attention due to its inherent privacy protection capability and heterogeneous collaboration.However

recent studies have revealed a potential privacy risk known as “gradient leakage”

where the gradients can be used to determine whether a data record with a specific property is included in another participant’s batch

thereby exposing the participant’s training data.Current privacy-enhanced federated learning methods may have drawbacks such as reduced accuracy

computational overhead

or new insecurity factors.To address this issue

a differential privacy-enhanced generative adversarial network model was proposed

which introduced an identifier into vanilla GAN

thus enabling the input data to be approached while satisfying differential privacy constraints.Then this model was applied to the federated learning framework

to improve the privacy protection capability without compromising model accuracy.The proposed method was verified through simulations under the client/server (C/S) federated learning architecture and was found to balance data privacy and practicality effectively compared with the DP-SGD method.Besides

the usability of the proposed model was theoretically analyzed under a peer-to-peer (P2P) architecture

and future research work was discussed.

关键词

Keywords

references

MCMAHAN H B , MOORE E , RAMAGE D , et al . Communication-efficient learning of deep networks from decentralized data [J ] . arXiv Preprint arXiv:1602.05629 , 2017 .

ZHU L , LIU Z , HAN S . Deep leakage from gradients [C ] // Advances in Neural Information Processing Systems . 2019 :32.

ZHAO B , MOPURI KR , BILEN H . IDLG:improved deep leakage from gradients [J ] . arXiv Preprint arXiv:2001.02610 , 2020 .

GEIPING J , BAUERMEISTER H , DRÖGE H , , et al . Inverting gradients:how easy is it to break privacy in federated learning [J ] . arXiv Preprint arXiv:2003.14053 , 2020 .

ZHAO Y , ZHAO J , YANG M , et al . Local differential privacy based federated learning for internet of things [J ] . arXiv Preprint arXiv:2004.08856 , 2020 .

LI Y , ZHOU Y P , JOLFAEI A , et al . Privacy-preserving federated learning framework based on chained secure multiparty computing [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 8 ): 6178 - 6186 .

TRUEX S , BARACALDO N , ANWAR A , et al . A hybrid approach to privacy-preserving federated learning [J ] . Informatik Spektrum , 2019 , 42 ( 5 ): 356 - 357 .

杨庚 , 王周生 . 联邦学习中的隐私保护研究进展 [J ] . 南京邮电大学学报(自然科学版) , 2020 , 40 ( 5 ): 204 - 214 .

YANG G , WANG Z S . Survey on privacy preservation in federated learning [J ] . Journal of Nanjing University of Posts and Telecommunications (Natural Science Edition) , 2020 , 40 ( 5 ): 204 - 214 .

陈兵 , 成翔 , 张佳乐 , 等 . 联邦学习安全与隐私保护综述 [J ] . 南京航空航天大学学报 , 2020 , 52 ( 5 ): 675 - 684 .

CHEN B , CHENG X , ZHANG J L , et al . Survey of security and privacy in federated learning [J ] . Journal of Nanjing University of Aeronautics ＆ Astronautics , 2020 , 52 ( 5 ): 675 - 684 .

MOHASSEL P , ZHANG Y . SecureML:a system for scalable privacy-preserving machine learning [C ] // 2017 IEEE Symposium on Security and Privacy (SP) . 2017 : 19 - 38 .

KILBERTUS N , GASCON A , KUSNER M , et al . Blind justice:fairness with encrypted sensitive attributes [C ] // Proceedings of the 35th International Conference on Machine Learning . 2018 : 2630 - 2639 .

HITAJ B , ATENIESE G , PEREZ-CRUZ F . Deep models under the gan:information leakage from collaborative deep learning [C ] // Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security . 2017 : 603 - 618 .

WANG Z B , SONG M K , ZHANG Z F , et al . Beyond inferring class representatives:user-level privacy leakage from federated learning [C ] // Proceedings of IEEE INFOCOM 2019-IEEE Conference on Computer Communications . 2019 : 2512 - 2520 .

HARDY C , LE MERRER E , SERICOLA B . MD-GAN:multi-discriminator generative adversarial networks for distributed datasets [C ] // Proceedings of 2019 IEEE International Parallel and Distributed Processing Symposium . 2019 : 866 - 877 .

RASOULI M , SUN T , RAJAGOPAL R . FedGAN:federated generative adversarial networks for distributed data [J ] . arXiv Preprint arXiv:2006.07228 , 2020 .

MUGUNTHAN V , GOKUL V , KAGAL L , et al . Bias-free FedGAN:a federated approach to generate bias-free datasets [J ] . arXiv Preprint arXiv:2103.09876 , 2021 .

ZHAO Z , BIRKE R , KUNAR A , et al . Fed-TGAN:federated learning framework for synthesizing tabular data [J ] . arXiv:2108.07927 , 2021 .

DWORK C , KENTHAPADI K , MCSHERRY F , et al . Our data,ourselves:privacy via distributed noise generation [C ] // Advances in Cryptology - Eurocrypt 2006 . 2006 : 486 - 503 .

DWORK C , MCSHERRY F , NISSIM K , et al . Calibrating noise to sensitivity in private data analysis [C ] // Theory of Cryptography . 2006 : 265 - 284 .

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial networks [J ] . Communications of the ACM , 2020 , 63 ( 11 ): 139 - 144 .

MELIS L , SONG C , DE CRISTOFARO E , et al . Exploiting unintended feature leakage in collaborative learning [C ] // 2019 IEEE Symposium on Security and Privacy (SP) . 2019 : 691 - 706 .

ABADI M , CHU A , GOODFELLOW I , et al . Deep learning with differential privacy [C ] // Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security . 2016 : 308 - 318 .

AUGENSTEIN S , MCMAHAN H B , RAMAGE D , et al . Generative models for effective ML on private,decentralized datasets [J ] . arXiv:1911.06679,B2020 .

SALIMANS T , GOODFELLOW I , ZAREMBA W , et al . Improved techniques for training GANs [J ] . arXiv Preprint arXiv:1606.03498 , 2016 .

HEUSEL M , RAMSAUER H , UNTERTHINER T , et al . GANs trained by a two time-scale update rule converge to a local Nash equilibrium [J ] . arXiv Preprint arXiv:1706.08500 , 2018 .

NGUYEN D C , DING M , PHAM Q V , et al . Federated learning meets blockchain in edge computing:opportunities and challenges [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 16 ): 12806 - 12825 .

Views

434

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

DP-FedDRC: differentially private personalized federated learning to alleviate dimension collapse

FLWD:A Webshell detection method based on federated learning

Survey on incentive-driven federated learning: privacy and security

Privacy-preserving federated learning framework with dynamic weight aggregation

Differentially private sequence generative adversarial networks for data privacy masking

Related Author

Wang Minli

Zhang Yuhao

Hu Changhui

ZENG Qingpeng

CHAI Jiangli

WU Shuixiu

CHI Huanhuan

XIONG Ping

Related Institution

School of Cyberspace Security (School of Cryptography), Hainan University

School of Mathematics and Computer Sciences, Nanchang University

School of Computer Information Engineering, Jiangxi Normal University

Faculty of Date Science, City University of Macau, Macau

School of Information Engineering, Zhongnan University of Economics and Law

⁰