| 摘要: |
| 联邦学习因能解决数据孤岛问题而被广泛关注, 但也存在用户隐私泄露风险和非独立同分布数据下模型异构导致性能下降的问题. 针对该问题, 提出基于Bregman散度和差分隐私的个性化联邦学习方法(FedBDP). 所提方法采用Bregman散度衡量本地参数与全局参数的差异, 并将其作为正则化项更新损失函数, 以减小模型差异来提升模型准确率. 同时, 采用自适应差分隐私技术对本地模型参数进行扰动, 通过定义衰减系数动态调整每轮差分隐私噪声的大小, 以合理分配隐私噪声大小并提升模型可用性. 理论分析表明FedBDP在强凸和非凸光滑函数下满足收敛条件. 实验结果验证该方法在满足差分隐私的前提下, FedBDP模型在MNIST和CIFAR10数据集下能够保证模型准确率. |
| 关键词: 隐私保护 个性化联邦学习 差分隐私 Bregman散度 |
| DOI：10.13328/j.cnki.jos.007032 |
| 分类号:TP306 |
| 基金项目:国家重点研发计划(2021YFB3101201); 国家自然科学基金(62272162, 62172159, 62172350, 62032020); 教育部人文社会科学研究规划基金(22YJAZH155); 湖南省自然科学基金(2023JJ30267) |
|
| Personalized Federated Learning Method Based on Bregman Divergence and Differential Privacy |
|
ZHANG Shao-Bo1, ZHANG Ji-Yong1, ZHU Geng-Ming1, LONG Sai-Qin2, LI Zhe-Tao2
|
|
1.School of Computer Science and Engineering, Hunan University of Science and Technology, Xiangtan 411201, China;2.College of Information Science and Technology, Jinan University, Guangzhou 510632, China
|
| Abstract: |
| Federated learning has caught much attention because it can solve data islands. However, it also faces challenges such as the risk of privacy leakage and performance degradation due to model heterogeneity under non-independent and identically distributed data. To this end, this study proposes a personalized federated learning method based on Bregman divergence and differential privacy (FedBDP). This method employs Bregman divergence to measure the differences between local and global parameters and adopt it as a regularization term to update the loss function, thereby reducing model differences to improve model accuracy. Meanwhile, adaptive differential privacy technology is utilized to perturb local model parameters, and the attenuation coefficient is defined to dynamically adjust the level of the differential privacy noise in each round, and thus reasonably allocate the privacy noise level and improve the model availability. Theoretical analysis shows that FedBDP satisfies convergence conditions under both strongly convex and non-convex smooth functions. Experimental results demonstrate that the FedBDP method can guarantee accuracy in the MNIST and CIFAR10 datasets on the premise of satisfying differential privacy. |
| Key words: privacy protection personalized federated learning differential privacy Bregman divergence |
