| 引用本文: | 朱渝珊,张文,王晓珂,李志宇,陈名杨,姚祯,陈辉,陈华钧.面向电子商务社交知识图谱高效增量预训练的双向模仿蒸馏.软件学报,2025,36(3):1218-1239 |
| |
|
| 本文已被：浏览 1110次 下载 3437次 |
码上扫一扫！ |
|
|
| 面向电子商务社交知识图谱高效增量预训练的双向模仿蒸馏 |
|
朱渝珊1, 张文2, 王晓珂3, 李志宇3, 陈名杨1, 姚祯2, 陈辉3, 陈华钧1
|
|
1.浙江大学 计算机科学与技术学院, 浙江 杭州 310012;2.浙江大学 软件学院, 浙江 宁波 315048;3.阿里巴巴集团, 浙江 杭州 311121
|
|
| 摘要: |
| 知识图谱(knowledge graph, KG)预训练模型有助于电子商务应用中各种下游任务, 然而, 对于具有高动态性的大规模电商社交知识图谱来说, 预训练模型需要及时更新以感知由用户交互引起的节点特征变化. 提出一种针对电商社交知识图谱预训练模型的高效增量学习方法, 该方法通过基于双向模仿蒸馏的训练策略充分挖掘不同样本对模型更新的作用, 并通过基于样本常规性和反常性的采样策略来减少训练数据规模, 提升模型更新效率. 此外, 还提出一种逆重放机制, 为社交知识图谱预训练模型的增量训练生成高质量的负样本. 在真实的电子商务数据集和相关下游任务上的实验结果表明, 相较于最先进的方法, 所提方法可以更有效且高效地增量更新社交知识图谱预训练模型. |
| 关键词: 知识图谱 知识图谱预训练 增量学习 知识蒸馏 |
| DOI：10.13328/j.cnki.jos.007170 |
| 分类号:TP181 |
| 基金项目:国家自然科学基金(62306276, U23B2055, U19B2027, 91846204); 浙江省自然科学基金(LQ23F020017); 宁波市自然科学基金(2023J291) |
|
| Bidirectional Imitation Distillation for Efficient Incremental Pre-training of E-commerce Social Knowledge Graph |
|
ZHU Yu-Shan1, ZHANG Wen2, WANG Xiao-Ke3, LI Zhi-Yu3, CHEN Ming-Yang1, YAO Zhen2, CHEN Hui3, CHEN Hua-Jun1
|
|
1.College of Computer Science and Technology, Zhejiang University, Hangzhou 310012, China;2.School of Software Technology, Zhejiang University, Ningbo 315048, China;3.Alibaba Group, Hangzhou 311121, China
|
| Abstract: |
| Pre-training knowledge graph (KG) models facilitate various downstream tasks in e-commerce applications. However, large-scale social KGs are highly dynamic, and the pre-training models need to be updated regularly to reflect the changes in node features caused by user interactions. This study proposes an efficient incremental update framework for the pre-training KG models. The framework mainly includes a bidirectional imitation distillation method to fully use the different types of facts in new data, and a sampling strategy based on samples’ normality and abnormality is proposed to sample the most valuable facts from all new facts to reduce the training data size, and a reverse replay mechanism is proposed to generate high-quality negative facts that are more suitable for the incremental training of social KGs in e-commerce. Experimental results on real-world e-commerce datasets and related downstream tasks demonstrate that the proposed framework can incrementally update the pre-training KG models more effectively and efficiently compared to state-of-the-art methods. |
| Key words: knowledge graph (KG) knowledge graph pre-training incremental learning knowledge distillation |
