| 本文已被：浏览 1850次 下载 5194次 |
码上扫一扫！ |
|
|
| 测试集有效性评价：问题、进展与挑战 |
|
路则雨1,2, 张鹏1,2, 王洋1,2, 郭肇强1,2, 杨已彪1,2, 周毓明1,2
|
|
1.计算机软件新技术国家重点实验室(南京大学), 江苏 南京 210023;2.南京大学 计算机科学与技术系, 江苏 南京 210023
|
|
| 摘要: |
| 测试用例集的缺陷检测有效性指测试集能够在多大程度上检测出软件中存在的缺陷. 如何评价测试集的缺陷检测有效性是一个重要问题. 覆盖率和变异得分是两个最重要和最广泛使用的测试集有效性度量. 为量化测试集的缺陷检测能力, 研究人员对测试集有效性评价进行了大量研究并且取得了较大的进展. 与此同时, 现有研究存在不一致的结论, 该领域依然存在一些亟待解决的挑战. 对多年来国内外学者在测试集有效性评价领域的研究成果进行系统性的梳理和总结. 首先, 阐述测试集有效性评价研究中的问题. 然后, 介绍并分析基于覆盖率和基于变异得分的测试集有效性的评价以及介绍测试集有效性评价在测试集优化中的应用. 最后, 指出测试集有效性评价研究中面临的挑战并给出建议的研究方向. |
| 关键词: 覆盖率 变异得分 缺陷检测 测试集有效性 |
| DOI：10.13328/j.cnki.jos.006953 |
| 分类号: |
| 基金项目:国家自然科学基金(62172205,62072194) |
|
| Evaluation of Test Suite Effectiveness: Problem, Progress, and Challenges |
|
LU Ze-Yu1,2, ZHANG Peng1,2, WANG Yang1,2, GUO Zhao-Qiang1,2, YANG Yi-Biao1,2, ZHOU Yu-Ming1,2
|
|
1.State Key Laboratory for Novel Software Technology(Nanjing University), Nanjing 210023, China;2.Department of Computer Science and Technology, Nanjing University, Nanjing 210023, China
|
| Abstract: |
| The effectiveness of a test suite in defect detection refers to the extent to which the test suite could detect the defects hidden in the software. How to evaluate this performance of a test suite is an important issue. Coverage and mutation score are two of the most important and widely used metrics for test suite effectiveness. To quantify the defect detection capability of a test suite, researchers have devoted a large amount of research effort to this issue and have made significant progress. However, inconsistent conclusions can be observed among the existing studies, and some challenges still call for prompt resolution in the area. This study systematically summarizes the research results achieved by scholars both in China and abroad in the field of the evaluation of test suite effectiveness over the years. To start with, it expounds the problems in the research on the evaluation of test suite effectiveness. Then, it outlines and analyzes the evaluation of test suite effectiveness based on coverage and mutation score and presents the application of the evaluation of test suite effectiveness in test suite optimization. Finally, the study points out the challenges faced by this line of research and suggests the directions of future research. |
| Key words: coverage mutation score defect detection test suite effectiveness |
