动态聚合权重的隐私保护联邦学习框架

应作斌; 方一晨; 张怡文

doi:10.11959/j.issn.2096-109x.2022069

您当前的位置：

首页 >

文章列表页 >

动态聚合权重的隐私保护联邦学习框架

专题：大数据与人工智能安全 | 更新时间：2024-06-03

- 动态聚合权重的隐私保护联邦学习框架
- Privacy-preserving federated learning framework with dynamic weight aggregation
- 网络与信息安全学报 2022年8卷第5期页码：56-65
- 作者机构：
  
  1. 澳门城市大学，中国澳门 999078
  2. 安徽新华学院，安徽合肥 230000
- 作者简介：
  
  [ "应作斌（1982- ），男，安徽芜湖人，澳门城市大学助理教授，主要研究方向为区块链、联邦学习" ]
  [ "方一晨（1998- ），女，浙江湖州人，澳门城市大学硕士生，主要研究方向为差分隐私、联邦学习" ]
  [ "张怡文（1980- ），女，安徽阜阳人，安徽新华学院教授，主要研究方向为数据挖掘、联邦学习" ]
- 基金信息：
  
  澳门科学技术发展基金;General R＆D Subsidy Program Fund Macau(0038/2022/A)
- DOI：10.11959/j.issn.2096-109x.2022069
  中图分类号： TP393
- 网络首发：2022-10，
  
  纸质出版：2022-10-15
- 稿件说明：
移动端阅览
应作斌, 方一晨, 张怡文. 动态聚合权重的隐私保护联邦学习框架[J]. 网络与信息安全学报, 2022,8(5):56-65.

Zuobin YING, Yichen FANG, Yiwen ZHANG. Privacy-preserving federated learning framework with dynamic weight aggregation[J]. Chinese Journal of Network and Information Security, 2022, 8(5): 56-65.
应作斌, 方一晨, 张怡文. 动态聚合权重的隐私保护联邦学习框架[J]. 网络与信息安全学报, 2022,8(5):56-65. DOI： 10.11959/j.issn.2096-109x.2022069.

Zuobin YING, Yichen FANG, Yiwen ZHANG. Privacy-preserving federated learning framework with dynamic weight aggregation[J]. Chinese Journal of Network and Information Security, 2022, 8(5): 56-65. DOI： 10.11959/j.issn.2096-109x.2022069.

摘要

在非可信中心服务器下的隐私保护联邦学习框架中，存在以下两个问题。① 在中心服务器上聚合分布式学习模型时使用固定的权重，通常是每个参与方的数据集大小。然而，不同参与方具有非独立同分布的数据，设置固定聚合权重会使全局模型的效用无法达到最优。② 现有框架建立在中心服务器是诚实的假定下，没有考虑中央服务器不可信导致的参与方的数据隐私泄露问题。为了解决上述问题，基于比较流行的DP-FedAvg算法，提出了一种非可信中心服务器下的动态聚合权重的隐私保护联邦学习DP-DFL框架，其设定了一种动态的模型聚合权重，该方法从不同参与方的数据中直接学习联邦学习中的模型聚合权重，从而适用于非独立同分布的数据环境。此外，在本地模型隐私保护阶段注入噪声进行模型参数的隐私保护，满足不可信中心服务器的设定，从而降低本地参与方模型参数上传中的隐私泄露风险。在数据集CIFAR-10 上的实验证明，DP-DFL 框架不仅提供本地隐私保证，同时可以实现更高的准确率，相较DP-FedAvg算法模型的平均准确率提高了2.09%。

Abstract

There are two problems with the privacy-preserving federal learning framework under an unreliable central server.① A fixed weight

typically the size of each participant’s dataset

is used when aggregating distributed learning models on the central server.However

different participants have non-independent and homogeneously distributed data

then setting fixed aggregation weights would prevent the global model from achieving optimal utility.② Existing frameworks are built on the assumption that the central server is honest

and do not consider the problem of data privacy leakage of participants due to the untrustworthiness of the central server.To address the above issues

based on the popular DP-FedAvg algorithm

a privacy-preserving federated learning DP-DFL algorithm for dynamic weight aggregation under a non-trusted central server was proposed which set a dynamic model aggregation weight.The proposed algorithm learned the model aggregation weight in federated learning directly from the data of different participants

and thus it is applicable to non-independent homogeneously distributed data environment.In addition

the privacy of model parameters was protected using noise in the local model privacy protection phase

which satisfied the untrustworthy central server setting and thus reduced the risk of privacy leakage in the upload of model parameters from local participants.Experiments on dataset CIFAR-10 demonstrate that the DP-DFL algorithm not only provides local privacy guarantees

but also achieves higher accuracy rates with an average accuracy improvement of 2.09% compared to the DP-FedAvg algorithm models.

关键词

Keywords

references

LI T , SAHU A K , TALWALKAR A , et al . Federated learning:challenges,methods,and future directions [J ] . IEEE Signal Processing Magazine , 2020 , 37 ( 3 ): 50 - 60 .

XU X H , PENG H , SUN L C , et al . Fedmood:federated learning on mobile health data for mood detection [J ] . arXiv preprint arXiv:2102.09342 , 2021 .

CHE SICONG , PENG H , SUN L C , et al . Federated multiview learning for private medical data integration and analysis [J ] . arXiv preprint arXiv:2105.01603 , 2021 .

MC-MAHAN B , MOORE E , RAMAGE D , et al . Communication-efficient learning of deep networks from decentralized data [C ] // Artificial Intelligence and Statistics . 2017 : 1273 - 1282 .

LIU Z , GUO J , YANG W , et al . Privacy-preserving aggregation in federated learning:a survey [J ] . arXiv preprint arXiv:2203.17005 , 2022 .

RAMASWAMY S , THAKKAR O , MATHEWS R , et al . Training production language models without memorizing user data [J ] . arXiv preprint arXiv:2009.10031 , 2020 .

LI Z , SHARMA V , MOHANTY S P . Preserving data privacy via federated learning:challenges and solutions [J ] . IEEE Consumer Electronics Magazine , 2020 , 9 ( 3 ): 8 - 16 .

LI T , SAHU A K , ZAHEER M , et al . Federated optimization in heterogeneous networks [J ] . Proceedings of Machine Learning and Systems , 2020 , 2 : 429 - 450 .

陈前昕 , 毕仁万 , 林劼 , 等 . 支持多数不规则用户的隐私保护联邦学习框架 [J ] . 网络与信息安全学报 , 2022 , 8 ( 1 ): 139 - 150 .

CHEN Q X , BI R W , LIN J , et al . Privacy-preserving federated learning framework with irregular-majority users [J ] . Chinese Journal of Network and Information Security , 2022 , 8 ( 1 ): 139 - 150 .

刘艺璇 , 陈红 , 刘宇涵 , 等 . 联邦学习中的隐私保护技术 [J ] . 软件学报 , 2021 , 33 ( 3 ): 1057 - 1092 .

LIU Y X , CHEN H , LIU Y H , et al . Privacy-preserving techniques in Federated Learning [J ] . Journal of Software , 2022 , 33 ( 3 ): 1057 - 1092 .

王腾 , 霍峥 , 黄亚鑫 , 等 . 联邦学习中的隐私保护技术研究综述 [J ] . 计算机应用 , 2022 .

WANG T , HUO Z , HUANG Y X , et al . Survey of privacy-preserving technologies in federated learning [J ] . Journal of Computer Applications , 2022 .

FIGURNOV M , MOHAMED S , MNIH A . Implicit repara- meterization gradients [C ] // Advances in Neural Information Processing Systems . 2018 : 441 - 452 .

PAPERNOT N , ABADI M , ERLINGSSON U F , et al . Semi-supervised knowledge transfer for deep learning from private training data [C ] // ICLR . 2017 .

SONG C , RISTENPART T , SHMATIKOV V . Machine learning models that remember too much [C ] // 2017 ACM SIGSAC Conf.on Computer and Communications Security . 2017 : 587 - 601 .

SHOKRI R , STRONATI M , SONG C , et al . Membership inference attacks against machine learning models [C ] // 2017 IEEE Symposium on Security and Privacy (SP) . 2017 : 3 - 18 .

KIM M , GÜNLÜ O , SCHAEFER R F . Federated learning with local differential privacy:Trade-offs between privacy,utility,and communication [C ] // ICASSP 2021-2021 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP) . 2021 : 2650 - 2654 .

SUN L , QIAN J , CHEN X . LDP-FL:practical private aggregation in federated learning with local differential privacy [J ] . arXiv preprint arXiv:2007.15789 , 2020 .

ABADI M , CHU A , GOODFELLOW I , et al . Deep learning with differential privacy [C ] // Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security . 2016 : 308 - 318 .

MIRONOV I , TALWAR K , ZHANG L . R'enyi differential privacy of the sampled gaussian mechanism [J ] . arXiv preprint arXiv:1908.10530 , 2019 .

SUN L , LYU L . Federated model distillation with noise-free differential privacy [J ] . arXiv preprint arXiv:2009.05537 , 2020 .

NAGAR A , TRAN C , FIORETTO F . Privacy-preserving and accountable multi-agent learning [C ] // AAMAS Conference proceedings . 2021 .

DAI Z , LOW B K H , JAILLET P . Differentially private federated Bayesian optimization with distributed exploration [J ] . Advances in Neural Information Processing Systems , 2021 ,34.

ANDREW G , THAKKAR O , MCMAHAN B , et al . Differentially private learning with adaptive clipping [J ] . Advances in Neural Information Processing Systems , 2021 ,34.

MC-MAHAN H B , RAMAGE D , TALWAR K , et al . Learning differentially private recurrent language models [C ] // arXiv preprint arXiv:1710 . 06963 , 2017 .

GEYER R C , KLEIN T , NABI M . Differentially private federated learning:a client level perspective [J ] . arXiv preprint arXiv:1712.07557 , 2017 .

WANG J Y , LIU Q H , LIANG H , et al . Tackling the objective inconsistency problem in heterogeneous federated optimization [C ] // NeurIPS . 2020 .

ZHENG Q , CHEN S , LONG Q , et al . Federated f-differential privacy [C ] // International Conference on Artificial Intelligence and Statistics . 2021 : 2251 - 2259 .

HU R , GUO Y X , LI H N , et al . Personalized federated learning with differential privacy [J ] . IEEE Internet of Things Journal,7(10):9530–9539 , 2020 .

CHEN X N , WANG R C , CHENG M H , et al . Drnas:dirichlet neural architecture search [J ] . arXiv preprint arXiv:2006.10355 , 2020 .

XIA Y , YANG D , LI W , et al . Auto-FedAvg:learnable federated averaging for multi-institutional medical image segmentation [J ] . arXiv preprint arXiv:2104.10195 , 2021 .

ZHAO Y , ZHAO J , YANG M , et al . Local differential privacy-based federated learning for internet of things [J ] . IEEE Internet of Things Journal , 2022 , 8 ( 11 ): 8836 - 8853 .

LI Y W , CHANG T H , CHI C Y . Secure federated averaging algorithm with differential privacy [C ] // 2020 IEEE 30th International Workshopon Machine Learning for Signal Processing (MLSP) , 2020 : 1 - 6 .

浏览量

822

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

DP-FedDRC：缓解维度坍塌的差分隐私个性化联邦学习

FLWD:基于联邦学习的Webshell检测方法

激励驱动的联邦学习研究综述：隐私与安全

基于生成对抗网络的隐私增强联邦学习方案

基于动态聚类和信誉评估的抗攻击联邦框架