| 摘要: |
| 智能装置设备产生的时序数据增长迅速，存在严重的数据质量问题.劣质时序数据质量管理和数据质量提升技术需求日益迫切.时序数据的有序时窗、行列关联等特点，为时序数据质量语义表达提出了挑战.提出了一种同时考虑时序数据在行与列上的数据依赖信息的数据质量规则，即时序否定约束TDC.研究了TDC的定义与构建方法，从时窗与多阶表达式运算这两个方面，对已有的数据质量规则体系进行表达力的扩展，并提出针对兼顾行列的时序数据质量规则挖掘方法.在真实时序数据集上开展大量实验，实验结果验证了该方法能够有效且高效地挖掘时序数据中隐藏的数据质量规则.对比实验的结果表明，该方法能够有效地对行与列上的关联信息进行谓词构造；在质量规则挖掘效果上优于单纯的行上约束挖掘方法以及单纯的列上约束挖掘方法. |
| 关键词: 数据质量管理 数据质量规则 时序数据管理 工业大数据 |
| DOI：10.13328/j.cnki.jos.006793 |
| 分类号: |
| 基金项目:国家自然科学基金（62232005，62202126）；国家重点研发计划（2021YFB3300502）；CCF-华为胡杨林基金数据库专项（CCF-HuaweiDB202204）；黑龙江省博士后资助项目（LBH-Z21137） |
|
| Time Series Data Quality Rules Discovery with Both Row and Column Dependencies |
|
DING Xiao-Ou1, LI Ying-Ze1, WANG Chen2, WANG Hong-Zhi1, LI Hao-Xuan1
|
|
1.School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China;2.National Engineering Research Center for Big Data Software (Tsinghua University), Beijing 100084, China
|
| Abstract: |
| Time series data generated by intelligent devices are growing rapidly and faced with serious data quality problems. The demand for time series data quality management and data quality improvement based on data repairing techniques is increasingly urgent. Time series data has the obvious characteristics about the ordered time window and strong associations between rows and columns. This brings much more challenges for the research of the data quality semantic expression of time series data. This study proposes the definition and the construction of time series data quality rules, which takes into account the association on both rows and columns. It extends the expression of the existing data quality rule systems in terms of time window and multi-order expression operation. In addition, the discovery method is proposed for time series data quality rules. Experiment results on real time series data sets verify that the proposed method can effectively and efficiently discover hidden data quality rules from time series data, showing that the proposed method has higher performance with the predicate construction of associated information on row and column, compared with the existing data quality rule discovery method. |
| Key words: data quality management, data quality constraint, time series data management industrial big data |
