Journal of Software:2017.28(8):1999-2009

(武汉光电国家实验室(华中科技大学), 湖北 武汉 430074;华中科技大学 计算机科学与技术学院, 湖北 武汉 430074)
High-Performance Distributed Storage System for Large-Scale High-Definition Video Data
CAO Shun-De,HUA Yu,FENG Dan,SUN Yuan-Yuan,ZUO Peng-Fei
(Wuhan National Laboratory for Optoelectronics(Huazhong University of Science and Technology), Wuhan 430074, China;School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074, China)
Chart / table
Similar Articles
Article :Browse 1925   Download 2344
Received:August 11, 2016    Revised:September 21, 2016
> 中文摘要: 通过对视频监控数据的特点和传统存储方案进行分析,提出一种高性能分布式存储系统解决方案.不同于传统的基于文件存储的方式,设计了一种逻辑卷结构,将非结构化的视频流数据以此结构进行组织并直接写入RAW磁盘设备,解决了传统存储方案中随机磁盘读写和磁盘碎片导致存储性能下降的问题.该方案将元数据组织为两级索引结构,分别由状态管理器和存储服务器管理,极大地减少了状态管理器需要管理元数据的数量,消除了性能瓶颈,并提供了精确到秒级的检索精度.此外,该方案灵活的存储服务器分组策略和组内互备关系使得存储系统具备容错能力和线性扩展能力.系统测试结果表明,该方案在成本低廉的PC服务器上实现了单台服务器可同时记录400路1080P视频流,写入速度是本地文件系统的2.5倍.
Abstract:This paper presents a high-performance distributed storage solution for video surveillance data via analyzing the characteristics of video surveillance data and conventional file storage solutions. This design proposes a logical volume structure rather than file-based storage to efficiently organize unstructured video stream data. The scheme hence directly writes these stream data into raw disk devices, to address the problem of storage performance decrease caused by the random access and disk fragmentation in traditional storage systems. A two-stage index strategy is also implemented to manage metadata by the state manager and storage servers, which significantly reduces the amount of metadata managed by the state manager, eliminates the performance bottlenecks, and provides the second-level video retrieval accuracy. Moreover, the design has the salient features of fault tolerance and linear scaling abilities with the help of the flexible storage server grouping policy and the mutual backup relationship in a storage group. Experimental results show that the solution can simultaneously record 400 ways of 1080P video streams with a single low-cost PC server, and the system's average write speed is 2.5 times faster compared with the local file systems.
文章编号:     中图分类号:    文献标志码:
基金项目:国家重点研发计划(2016YFB1000202) 国家重点研发计划(2016YFB1000202)
Foundation items:National Key Research and Development Program of China (2016YFB1000202)
Reference text:


CAO Shun-De,HUA Yu,FENG Dan,SUN Yuan-Yuan,ZUO Peng-Fei.High-Performance Distributed Storage System for Large-Scale High-Definition Video Data.Journal of Software,2017,28(8):1999-2009