DP-FedDRC: differentially private personalized federated learning to alleviate dimension collapse

Wang Minli; Zhang Yuhao; Hu Changhui

doi:10.11959/j.issn.2096-109x.2026008

您当前的位置：

首页 >

文章列表页 >

DP-FedDRC: differentially private personalized federated learning to alleviate dimension collapse

Papers | 更新时间：2026-05-26

- DP-FedDRC: differentially private personalized federated learning to alleviate dimension collapse
- Chinese Journal of Network and Information Security Vol. 12, Issue 1, Pages: 104-116(2026)
- 作者机构：
  
  海南大学网络空间安全学院（密码学院），海南海口 570228
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(62262012);China Scholarship Council Funding(202407560059);Hainan Provincial Natural Science Foundation of China(124QN176);Hainan Postgraduate Innovative Scientific Research Project(Qhyb2024-02);Start-up Research Foundation of Hainan University(KYQD22094)
- DOI：10.11959/j.issn.2096-109x.2026008
  CLC： TP309
- Received：30 July 2025，
  
  Revised：2025-11-09，
  
  Published：25 February 2026
- 稿件说明：
移动端阅览
王民利, 张宇浩, 胡昌慧. DP-FedDRC：缓解维度坍塌的差分隐私个性化联邦学习[J]. 网络与信息安全学报, 2026, 12(1): 104-116.

Wang Minli, Zhang Yuhao, Hu Changhui. DP-FedDRC: differentially private personalized federated learning to alleviate dimension collapse[J]. Chinese Journal of Network and Information Security, 2026, 12(1): 104-116.
王民利, 张宇浩, 胡昌慧. DP-FedDRC：缓解维度坍塌的差分隐私个性化联邦学习[J]. 网络与信息安全学报, 2026, 12(1): 104-116. DOI： 10.11959/j.issn.2096-109x.2026008.

Wang Minli, Zhang Yuhao, Hu Changhui. DP-FedDRC: differentially private personalized federated learning to alleviate dimension collapse[J]. Chinese Journal of Network and Information Security, 2026, 12(1): 104-116. DOI： 10.11959/j.issn.2096-109x.2026008.

摘要

联邦学习在分布式隐私保护场景中面临数据异质性与隐私泄露双重挑战。个性化联邦学习通过约束局部模型更新方向缓解数据分布差异，而差分隐私则提供严格的隐私保障。然而，现有方法在同时解决这两个问题时，未能充分考量噪声对数据非独立同分布场景下联邦学习训练过程的耦合影响，导致隐私-效用权衡不足。实验表明，差分隐私噪声会显著加剧数据异质性引发的维度坍塌问题。该问题导致特征空间的有效自由度衰减，并削弱了特征的判别性表达能力，成为模型性能下降的关键原因。为此，提出一种融合特征去相关与特征残差约束的差分隐私个性化联邦学习框架（DP-FedDRC）。该框架通过抑制特征冗余相关性并保留关键判别信息，缓解噪声和数据异质性共同导致的维度坍塌问题。在隐私保护方面，DP-FedDRC在本地训练中采用DP-SGD机制实现梯度裁剪与高斯噪声注入，并基于Rényi差分隐私框架严格核算全局隐私预算。在MNIST、Fashion-MNIST和CIFAR-10数据集上的实验表明，在同等严格的隐私约束下，DP-FedDRC相比基线算法实现了更高的模型准确率，且计算开销仅增加8.3%~9.9%。

Abstract

Federated learning faces dual challenges of data heterogeneity and privacy leakage in distributed privacy-preserving scenarios. Personalized federated learning mitigates data distribution discrepancies by constraining local model update directions

while differential privacy provides rigorous privacy guarantees. However

when solving these two problems at the same time

the existing methods fail to fully consider the coupling effect of noise on the federated learning and training process in the data Non-IID scenario

resulting in insufficient privacy-utility trade-offs. Experiments show that differential privacy noise significantly exacerbates the dimension collapse problem induced by data heterogeneity. This problem leads to the decay of effective degrees of freedom in the feature space and impairs discriminative feature representation

which is a key factor in model performance degradation. To address this

a differential privacy personalized federated learning framework incorporating feature decorrelation and residual constraints (DP-FedDRC) was proposed. This framework alleviates the dimension collapse problem caused by both noise and data heterogeneity by suppressing redundant feature correlations and preserving critical discriminative information. In terms of privacy protection

DP-FedDRC implements gradient clipping and Gaussian noise injection using the DP-SGD mechanism during local training

and rigorously accounts for the global privacy budget based on the Rényi differential privacy framework. Experiments on the MNIST

Fashion-MNIST

and CIFAR-10 datasets demonstrated that under equally strict privacy constraints

DP-FedDRC achieved higher model accuracy compared to baseline algorithms

with only an 8.3%~9.9% increase in computational overhead.

关键词

Keywords

references

Mcmahan H B , Moore E , Ramage D , et al . Communication-efficient learning of deep networks from decentralized data [C ] // Proceedings of the International Conference on Artificial Intelligence and Statistics . 2016 .

Yang Q , Liu Y , Chen T J , et al . Federated machine learning: concept and applications [J ] . ACM Transactions on Intelligent Systems and Technology , 2019 , 10 ( 2 ): 1 - 19 .

Kairouz P , Mcmahan H B . Advances and open problems in federated learning [J ] . Foundations and Trends in Machine Learning , 2021 , 14 ( 1 ): 1 - 210 .

Zhao Y , Li M , Lai L Z , et al . Federated learning with non-IID data [J ] . 2018 , arXiv: 1806.00582 .

周传鑫 , 孙奕 , 汪德刚 , 等 . 联邦学习研究综述 [J ] . 网络与信息安全学报 , 2021 , 7 ( 5 ): 77 - 92 .

Zhou C X , Sun Y , Wang D G , et al . A summary of federal learning research [J ] . Chinese Journal of Network and Information Security , 2021 , 7 ( 5 ): 77 - 92 .

高莹 , 陈晓峰 , 张一余 , 等 . 联邦学习系统攻击与防御技术研究综述 [J ] . 计算机学报 , 2023 , 46 ( 9 ): 1781 - 1805 .

Gao Y , Chen X F , Zhang Y Y , et al . Summary of research on attack and defense technology of federated learning system [J ] . Chinese Journal of Computers , 2023 , 46 ( 9 ): 1781 - 1805 .

Shokri R , Stronati M , Song C Z , et al . Membership inference attacks against machine learning models [C ] // Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP) . Piscataway : IEEE Press , 2017 : 3 - 18 .

Suri A , Kanani P , Marathe V J , et al . Subject membership inference attacks in federated learning [J ] . 2022 , arXiv: 2206.03317 .

Fredrikson M , Jha S , Ristenpart T . Model inversion attacks that exploit confidence information and basic countermeasures [C ] // Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security . New York : ACM , 2015 : 1322 - 1333 .

Wang X H , Wang S , Li Y W , et al . Differentially private and heterogeneity-robust federated learning with theoretical guarantee [J ] . IEEE Transactions on Artificial Intelligence , 2024 , 5 ( 12 ): 6369 - 6384 .

Liang J T , Su S . DP-FedEwc: Differentially private federated elastic weight consolidation for model personalization [J ] . Knowledge-Based Systems , 2024 , arXiv: 303. 112401 .

Shi Y J , Liang J , Zhang W Q , et al . Towards understanding and mitigating dimensional collapse in heterogeneous federated learning [C ] // Proceedings of the 11th International Conference on Learning Representations (ICLR) . 2023 : 9820 - 9843 .

Abadi M , Chu A , Goodfellow I , et al . Deep learning with differential privacy [C ] // Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security . New York : ACM , 2016 : 308 - 318 .

Li T , Sahu A K , Zaheer M , et al . Federated optimization in heterogeneous networks [C ] // Proceedings of Machine Learning and Systems . 2020 , 2 : 429 - 450 .

Karimireddy S P , Kale S , Mohri M , et al . SCAFFOLD: stochastic controlled averaging for federated learning [C ] // Proceedings of the 37th International Conference on Machine Learning . 2020 : 5132 - 5143 .

Durmus A E , Yue Z , Ramon M , et al . Federated learning based on dynamic regularization [C ] // Proceedings of the 9th International Conference on Learning Representations (ICLR) . 2021 : 790 - 825 .

Asad M , Moustafa A , Ito T . FedOpt: towards communication efficiency and privacy preservation in federated learning [C ] // Applied Sciences , 2020 , 10 ( 8 ): 2864 .

Wang S , Xu Y Q , Wang Z G , et al . Beyond ADMM: a unified client-variance-reduced adaptive federated learning framework [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2023 , 10175 - 10183 .

Girgis A , Data D , Diggavi S , et al . Shuffled model of differential privacy in federated learning [C ] // Proceedings of the 24th International Conference on Artificial Intelligence and Statistics . 2021 : 2521 - 2529 .

Wei W Q , Liu L , Wu Y Z , et al . Gradient-leakage resilient federated learning [C ] // Proceedings of the 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS) . Piscataway : IEEE Press , 2021 : 797 - 807 .

Fu J , Chen Z L , Han X . Adap DP-FL: differentially private federated learning with adaptive noise [C ] // Proceedings of the 2022 IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) . Piscataway : IEEE Press , 2023 : 656 - 663 .

Chen L , Ding X F , Bao Z F , et al . Differentially private federated learning on non-iid data: convergence analysis and adaptive optimization [J ] . IEEE Transactions on Knowledge and Data Engineering , 2024 , 36 ( 9 ): 4567 - 4581 .

Wei K , Li J , Ding M , et al . Federated learning with differential privacy: algorithms and performance analysis [J ] . IEEE Transactions on Information Forensics and Security , 2020 , 15 : 3454 - 3469 .

Noble M , Bellet A , Dieuleveut A . Differentially private federated learning on heterogeneous data [C ] // Proceedings of the 25th International Conference on Artificial Intelligence and Statistics . 2022 : 10110 - 10145 .

Li Y W , Chang T H , Chi C Y . Secure federated averaging algorithm with differential privacy [C ] // Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing (MLSP) . Piscataway : IEEE Press , 2020 : 1 - 6 .

Feng J , Yang L T , Ren B C , et al . Tensor recurrent neural network with differential privacy [J ] . IEEE Transactions on Computers , 2024 , 73 ( 3 ): 683 - 693 .

Zhao P N , Zhang Z K , Dong J W , et al . An attack-agnostic defense framework against manipulation attacks under local differential privacy [C ] // Proceedings of the 2025 IEEE Symposium on Security and Privacy (SP) . Piscataway : IEEE Press , 2025 : 3858 - 3876 .

Zhang P F , Sun H , Zhang Z K , et al . Privacy-preserving recommendations with mixture model-based matrix factorization under local differential privacy [J ] . IEEE Transactions on Industrial Informatics , 2025 , 21 ( 7 ): 5451 - 5459 .

Dwork C , Roth A . The algorithmic foundations of differential privacy [M ] . Hanover : Now Publishing , 2014 : 211 - 407 .

Mironov I . Rényi differential privacy [C ] // Proceedings of the 2017 IEEE 30th Computer Security Foundations Symposium (CSF) . Piscataway : IEEE Press , 2017 : 263 - 275 .

Cogswell M , Ahmed F , Girshick R , et al . Reducing overfitting in deep networks by decorrelating representations [J ] . 2015 , arXiv: 1511.06068 .

Xiong W , Du B , Zhang L F , et al . Regularizing deep convolutional neural networks with a structured decorrelation constraint [C ] // Proceedings of the 2016 IEEE 16th International Conference on Data Mining (ICDM) . Piscataway : IEEE Press , 2017 : 519 - 528 .

Wang Y X , Balle B , Kasiviswanathan S P . Subsampled Rényi differential privacy and analytical moments accountant [C ] // Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics . 2019 : 1226 - 1235 .

Lecun Y , Bottou L , Bengio Y , et al . Gradient-based learning applied to document recognition [J ] . Proceedings of the IEEE , 1998 , 86 ( 11 ): 2278 - 2324 .

Xiao H , Rasul K , Vollgraf R . Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms [J ] . 2017 , arXiv: 1708.07747 .

Krizhevsky A , Hinton G . Learning multiple layers of features from tiny images [R ] . Toronto : University of Toronto , 2009 .

Papernot N , Thakurta A , Song S , et al . Tempered sigmoid activations for deep learning with differential privacy [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 10 ): 9312 - 9321 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

FLWD:A Webshell detection method based on federated learning

Survey on incentive-driven federated learning: privacy and security

Privacy-enhanced federated learning scheme based on generative adversarial networks

Privacy-preserving federated learning framework with dynamic weight aggregation

Anti-attack federated framework based on dynamic clustering and reputation evaluation

Related Author

ZENG Qingpeng

CHAI Jiangli

WU Shuixiu

CHI Huanhuan

XIONG Ping

LIU Hengzhu

MA Xiao

ZHU Tianqing

Related Institution

School of Computer Information Engineering, Jiangxi Normal University

School of Mathematics and Computer Sciences, Nanchang University

School of Information Engineering, Zhongnan University of Economics and Law

Faculty of Date Science, City University of Macau, Macau

Zhicheng College, Fuzhou University

⁰