| 本文已被：浏览 2567次 下载 6201次 |
码上扫一扫！ |
|
|
| 面向大数据分析的分布式矩阵计算系统研究进展 |
|
陈梓浩1,2, 徐辰1,2,3, 钱卫宁1,2, 周傲英1,2
|
|
1.华东师范大学 数据科学与工程学院, 上海 200062;2.上海市大数据管理系统工程研究中心(华东师范大学), 上海 200062;3.广西可信软件重点实验室(桂林电子科技大学), 广西 桂林 541004
|
|
| 摘要: |
| 在大数据治理应用中，数据分析是必不可少的一环，且具有耗时长、计算资源需求大的特点，因此，优化其执行效率至关重要.早期由于数据规模不大，数据分析师可以利用传统的矩阵计算工具执行分析算法，然而随着数据量的爆炸式增长，诸如MATLAB等传统工具已无法满足应用需求的执行效率，进而涌现出了一批面向大数据分析的分布式矩阵计算系统.从技术、系统等角度综述了分布式矩阵计算系统的研究进展.首先，从发展成熟的数据管理领域的视角出发，剖析分布式矩阵计算系统在编程接口、编译优化、执行引擎、数据存储这4个层面面临的挑战；其次，分别就这4个层面展开，探讨、总结相关技术；最后，总体分析了典型的分布式矩阵计算系统，并展望了未来研究的发展方向. |
| 关键词: 大数据分析 矩阵计算 并行计算系统 |
| DOI：10.13328/j.cnki.jos.006785 |
| 分类号: |
| 基金项目:国家自然科学基金（61902128）；广西可信软件重点实验室研究课题 |
|
| Research Progress on Distributed Matrix Computation Systems for Big Data Analysis |
|
CHEN Zi-Hao1,2, XU Chen1,2,3, QIAN Wei-Ning1,2, ZHOU Ao-Ying1,2
|
|
1.School of Data Science and Engineering, East China Normal University, Shanghai 200062, China;2.Shanghai Engineering Research Center of Big Data Management (East China Normal University), Shanghai 200062, China;3.Guangxi Key Laboratory of Trusted Software (Guilin University of Electronic Technology), Guilin 541004, China
|
| Abstract: |
| As an essential part of big data governance applications, data analysis is characterized by time-consuming and large hardware requirements, making it essential to optimize its execution efficiency. Earlier, data analysts could execute analysis algorithms using traditional matrix computation tools. However, with the explosive growth of data volume, the traditional tools can no longer meet the performance requirements of applications. Hence, distributed matrix computation systems for big data analysis have emerged. This study reviews the progress of distributed matrix computation systems from technical and system perspectives. First, this study analyzes the challenges faced by distributed matrix computation systems in four dimensions:programming interface, compilation optimization, execution engine, and data storage, from the perspective of the mature data management field. Second, this study discusses and summarizes the technologies in each of these four dimensions. Finally, the study investigates the future research and development directions of distributed matrix computation systems. |
| Key words: big data analysis matrix computation parallel computation system |
