基于规则与学习的变异技术对比研究
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

TP311

基金项目:

国家重点研发计划(2022YFB4501902)、国家自然科学基金(62232001)


On the Experimental Comparsion of Rule-Based and Learning-Based Mutation Techniques
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    变异测试是一种有效的软件测试技术,通过生成变异体来模拟软件缺陷,帮助提升现有测试套件的缺陷检测能力.变异体的质量对于变异测试的有效性具有显著影响.传统的变异测试方法通常采用人工设计的基于语法规则的变异算子生成变异体,并已取得一定的研究成果.近年来,许多研究开始结合深度学习技术,通过学习开源项目历史代码生成变异体.目前该新方法在变异体生成方面取得了初步的成果.基于语法规则和基于学习的两种变异技术,其机理不同,但其目标均是通过生成变异体来提高测试套件的缺陷检测能力,因此全面比较这两种变异技术对于变异测试及其下游任务(如缺陷定位和修复)至关重要.针对这一问题,本文设计实现一项针对基于语法规则和基于学习的变异技术的实证研究,旨在了解不同机理的变异技术在变异测试任务上的性能,以及生成的变异体在程序语义上的差异性.具体地,本文以Defect4J v1.2.0数据集为实验对象,比较MAJOR和PIT为代表的基于语法规则的变异技术和以DeepMutation、μBERT和LEAM为代表的基于深度学习的变异技术.实验结果表明,基于规则与学习的变异技术均可有效支持变异测试实践,但MAJOR的测试效果最优,能够检测85.4%的真实缺陷.在语义表示上,MAJOR具有最强的语义代表能力,基于其构造的测试套件能够杀死其余变异技术生成的超过95%占比的变异体.在缺陷表征上,两类技术均具有独特性.例如,基于规则的技术具有更稳定的代码元素变异能力,而基于学习的技术具有更强的上下文理解能力.

    Abstract:

    Mutation testing is an effective software testing technique. It helps improve the defect detection capability of an existing test suite by generating mutants that simulate software defects. The quality of mutants has a significant impact on the effectiveness of mutation testing. The traditional mutation testing approach typically employs manually designed syntactic rule-based mutation operators to generate mutants, and has achieved some academic success. In recent years, many studies have started to incorporate deep learning techniques to generate mutants by learning historical code from open source projects. This new approach has achieved preliminary progress in mutant generation. A comprehensive comparison of the two mutation techniques, i.e. rule-based and learning-based, which have different mechanisms but both aim to improve the defect detection capability of the test suite by mutation, is crucial for mutation testing and its downstream tasks (e.g., defect localisation and repair). To handle the problem, this paper designs and implements an empirical study of rule-based and learning-based mutation techniques, aiming to understand the performance of mutation techniques with different mechanisms on the task of mutation testing, as well as the variability of the generated mutants in terms of program semantics. Specifically, this paper uses the Defect4J v1.2.0 dataset to compare the syntactic rule-based mutation techniques represented by MAJOR and PIT with the deep learning-based mutation techniques represented by DeepMutation, μBERT and LEAM. The experimental results show that both rule-based and learning-based mutation techniques can effectively support mutation testing practices, but MAJOR has the best testing performance and is able to detect 85.4% of real defects. In terms of semantic representation, MAJOR has the strongest semantic representation capability, and its constructed test suite is able to kill more than 95% of the mutants generated by other mutation techniques. In terms of defect representation, both types of techniques are unique. For example, the rule-based technique has a more stable ability to mutate code elements, while the learning-based technique has a stronger ability to understand the context.

    参考文献
    相似文献
    引证文献
引用本文

贡志豪,陈逸洲,陈俊洁,郝丹.基于规则与学习的变异技术对比研究.软件学报,2024,35(7):0

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2023-09-11
  • 最后修改日期:2023-10-30
  • 录用日期:
  • 在线发布日期: 2024-01-05
  • 出版日期:
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号