大容量高并发下数据库稳定性优化研究
CSTR:
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

TP311

基金项目:

国家重点研发计划(2023YFB4503600); 国家自然科学基金联合基金(U22A2025); 华为创新技术合作项目(TC20220317022, TC20250526018-2025-02)


Research on Database Stability Optimization Under Large-capacity and High-concurrency Load
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    数据库系统作为大数据基础设施的关键支撑, 其性能表现直接影响着上层应用的服务质量. 近年来随着新型存储硬件的不断发展, 数据库系统在应对大容量高并发场景时暴露出明显的稳定性缺陷, 实际测试发现, 数据库在大规模负载下性能劣化极其严重, 吞吐量普遍较低且出现失稳现象. 通过对运行过程中关键指标的监控和分析, 将问题定位到数据库的I/O读写模型, 认为I/O流程的缺陷导致数据库在大规模负载下刷脏不及时, 业务线程无干净页可用是引起稳定性下降的根本原因. 以缓冲区为对象将数据库I/O抽象为生产者-消费者模型, 分析了该模型存在的功能耦合问题, 据此提出了新的功能解耦I/O模型, 对数据库的刷脏机制和干净页产出机制进行了深度优化以提高干净页的产出效率, 并将改进后的NSGA-II算法应用到刷脏场景中用于多目标白盒化参数调优. 最后使用TPC-C和sysbench这2种常用的基准测试, 从数据规模、测试时间、并发数、读写模式等维度, 结合消融实验对该方案进行全面评估, 实验表明对于事务执行的平均吞吐量、稳定性、延迟等指标而言, 提出的整体方案相比基线方案以及其他优化方案均取得了明显优化效果.

    Abstract:

    Database systems serve as critical infrastructure for big data, with their performance directly affecting the quality of service (QoS) for upper-layer applications. With the rapid advancement of storage hardware technologies, traditional database systems increasingly exhibit stability challenges under large-scale and high-concurrency workloads. Baseline evaluations reveal severe performance degradation under intensive workloads, characterized by sharp throughput decline and noticeable jitter. Analysis of key operational metrics identifies the root cause of instability in the database I/O read-write model, where delayed dirty page flushing and insufficient candidate pages for backend threads reduce overall system stability. To address these issues, this study abstracts the database I/O mechanism into a producer-consumer model centered on buffer management and identifies inherent functional coupling problems. A novel functionally decoupled I/O model is proposed, featuring optimizations to the dirty page flushing mechanism and candidate page allocation strategy, thereby enhancing the supply of clean pages. Furthermore, the improved NSGA-II algorithm is integrated into the flushing framework for multi-objective white-box parameter tuning. Comprehensive evaluations are conducted using TPC-C and sysbench benchmarks across multiple dimensions, including data scale, testing duration, concurrency levels, and read-write patterns, supplemented by ablation studies. Experimental results demonstrate that the proposed framework achieves significant improvements over baseline approaches and existing optimization approaches in throughput, stability, and latency under high-pressure scenarios.

    参考文献
    相似文献
    引证文献
引用本文

胡谱赟,潘巍,马泽琪,韩宇翊,沈阳,李战怀.大容量高并发下数据库稳定性优化研究.软件学报,,():1-24

复制
相关视频

分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2025-08-01
  • 最后修改日期:2025-10-27
  • 录用日期:
  • 在线发布日期: 2026-04-29
  • 出版日期:
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号