| 本文已被：浏览 2998次 下载 6881次 |
码上扫一扫！ |
|
|
| 一种监控系统的链路跟踪型日志数据的存储设计 |
|
尤勇1, 汪浩2,3, 任天1, 顾胜晖2,3, 孙佳林1
|
|
1.美团点评, 上海 200335;2.南京大学软件学院, 江苏 南京 210023;3.计算机软件新技术国家重点实验室(南京大学), 江苏 南京 210023
|
|
| 摘要: |
| 随着软件系统越来越复杂化和分布化，为系统提供具有完善功能的监控服务显得越来越重要.APM（application performance management）系统通过采集软件系统运行时的各项指标数据来分析软件的运行状态，例如CPU、内存使用率、垃圾回收的耗时、QPS等指标.此外，APM系统也会在软件运行中生成各种日志数据.通常来说，它能提供的监控数据分为3种：指标统计数据、链路跟踪（tracing）数据以及离散事件记录.这些数据有助于系统或者服务的维护人员理解运行状态，从而确保系统或者服务的稳定运行.基于开源的APM监控系统——CAT系统，提出了一种针对tracing类型数据的存储设计方案，通过内存块批量写入的方式提升存储效率，并设计了两级索引以提高查询效率.从线上的真实运行数据来看，该方案在写入性能和查询性能方面均有较好的表现. |
| 关键词: 监控系统 日志存储 两级索引 |
| DOI：10.13328/j.cnki.jos.006234 |
| 分类号: |
| 基金项目:国家自然科学基金（62072227，61802173）；国家重点研发计划（2019YFE0105500）；江苏省政府间双边创新项目（BZ2020017）；计算机软件新技术国家重点实验室（南京大学）创新项目（ZZKT2019B01） |
|
| Storage Design of Tracing-logs for Application Performance Management System |
|
YOU Yong1, WANG Hao2,3, REN Tian1, GU Sheng-Hui2,3, SUN Jia-Lin1
|
|
1.Meituan-Dianping Group, Shanghai 200335, China;2.Software Institute, Nanjing University, Nanjing 210023, China;3.State Key Laboratory for Novel Software Technology(Nanjing University), Nanjing 210023, China
|
| Abstract: |
| With the software system becoming more and more complex and distributed, it is more and more important to provide monitoring services with complete functions for the system. APM (application performance management) system analyzes the running state of software by collecting various indicator data of software system, such as CPU, memory utilization, the consuming time of garbage collection, QPS. In addition, the APM system can also generate various types of logs during the operation of the software. Generally speaking, it can provide three types of monitoring data:statistic metrics, tracing data, and discrete event records. The data can help the maintenance personnel of the system or service understand the running state, so as to ensure the stable operation of the system or service. Based on the open-source APM monitoring system (i.e., CAT system), this study proposes a storage design scheme for tracing data. It improves the storage efficiency by memory block which is designed for batch writing logs, and query efficiency by the structure of the two-level index. Through analyzing the real on-line running data, the proposed scheme has sound performance in both write performance and query performance. |
| Key words: monitoring system log storage two-level indexing |
