| 摘要: |
| 随着人工智能时代的到来，图嵌入技术被越来越多地用来挖掘图中的信息.然而，现实生活中的图通常很大，因此，分布式图嵌入技术得到了广泛的关注.分布式图嵌入算法面临着两大难点：（1）图嵌入算法多种多样，没有一个通用的框架能够描述大部分的算法；（2）现在的分布式图嵌入算法扩展性不足，当处理大图时性能较低.针对以上两个挑战，首先提出一个通用的分布式图嵌入框架，具体地，将图嵌入算法中的采样流程和训练流程进行解耦，使得框架能够较好地表达多种不同的算法；其次，提出一种基于参数服务器的模型切分嵌入策略，具体地，将模型分别切分到计算节点和参数服务器上，同时使用数据洗牌的操作保证计算节点之间没有模型交互，从而减少了分布式计算中的通信开销.基于参数服务器实现了一种原型系统，并且用充分的实验证明了在不损失精度的前提下，基于模型切分的策略能够比基线系统取得更好的性能. |
| 关键词: 分布式机器学习 图嵌入 网络优化 |
| DOI：10.13328/j.cnki.jos.006186 |
| 分类号: |
| 基金项目:国家重点研发计划（2018YFB1004403）；国家自然科学基金（61832001）；北京大学-腾讯协同创新实验室项目 |
|
| Distributed Optimization and Implementation of Graph Embedding Algorithms |
|
ZHANG Wen-Tao1,2, YUAN Bin1, ZHANG Zhi-Peng1, CUI Bin1
|
|
1.Key Laboratory of High Confidence Software Technologies of Ministry of Education(Peking University), Beijing 100871, China;2.Department of Data Platform, Tencent Technology(Beijing) Co., Ltd., Beijing 100193, China
|
| Abstract: |
| With the advent of artificial intelligence, graph embedding techniques are more and more used to mine the information from graphs. However, graphs in real world are usually large and distributed graph embedding is needed. There are two main challenges in distributed graph embedding. (1) There exist many graph embedding methods and there is not a general framework for most of the embedding algorithms. (2) Existing distributed implementations of graph embedding suffer from poor scalability and perform bad on large graphs. To tackle the above two challenges, a general framework is firstly presented for distributed graph embedding. In detail, the process of sampling and training is separated in graph embedding such that the framework can describe different graph embedding methods. Second, a parameter server-based model partitioning strategy is proposed—the model is partitioned to both workers and servers and shuffling is used to ensure that there is no model exchange among workers. A prototype system is implemented on parameter server and solid experiments are conducted to show that partitioning-based strategy can get better performance than all baseline systems without loss of accuracy. |
| Key words: distributed machine learning graph embedding network optimization |
