| 摘要: |
| 作为人工智能工程化的实现工具, 智能计算框架已在近年来被广泛应用, 其可靠性对于人工智能的有效实现至关重要. 然而, 智能计算框架的可靠性保障具有挑战性, 一方面, 智能计算框架代码迭代迅速、测试困难; 另一方面, 与传统软件不同, 智能计算框架涉及大量张量计算, 其代码规范缺乏软件工程理论指导. 为了解决这一问题, 现有的工作主要使用模糊测试手段实现缺陷定位, 然而, 这类方法只能实现特定类型缺陷的精准定位, 却难以即时地在开发过程中引导开发者关注软件质量. 因此, 将国内外常见的智能计算框架(TensorFlow, 百度飞桨等)作为研究对象, 选取多种变更特征构建数据集, 在代码提交级别对智能计算框架进行即时缺陷预测. 另外, 在此基础上使用LDA主题建模技术挖掘代码和代码提交信息作为新的特征, 并使用随机森林进行预测. 结果发现AUC-ROC平均值为0.77, 且语义信息可以略微提升预测性能. 最后, 使用可解释机器学习方法SHAP分析各特征属性对模型预测输出的影响, 发现: (1)基本特征对于模型的影响符合传统软件开发规律; (2)代码和提交信息中的语义特征对模型的预测结果有重要影响; (3)不同系统中的不同特征对模型预测输出的贡献度排序也存在较大差异. |
| 关键词: 智能计算框架 即时缺陷预测 可解释人工智能 实证软件工程 |
| DOI：10.13328/j.cnki.jos.006874 |
| 分类号: |
| 基金项目:国家自然科学基金(61772200); 上海市自然科学基金(21ZR1416300) |
|
| Just-in-time Defect Prediction for Intelligent Computing Frameworks |
|
GE Jian, YU Hui-Qun, FAN Gui-Sheng, TANG Jian-Hao, HUANG Zi-Jie
|
|
School of Computer Science and Engineering, East China University of Science and Technology, Shanghai 200237, China
|
| Abstract: |
| In recent years, intelligent computing frameworks have been widely applied as implementation tools in artificial intelligence (AI) engineering, and the reliability of the frameworks is the key to AI implementation effectiveness. However, the reliability assurance of intelligent computing frameworks is challenging. On one hand, the code iteration of frameworks is fast, with difficult code testing. On the other hand, unlike traditional software, intelligent computing frameworks involve a large number of tensor calculations, and the code specification lacks the guidance of software engineering theory. To this end, existing research mostly employs fuzzy testing to localize defects. However, such a method can only accurately discover specific fault types, and it is difficult to guide developers and make them focus on software quality during the development process. Therefore, this study takes the popular intelligent computing frameworks (TensorFlow, Baidu PaddlePaddle, etc.) as the research object, selects multiple change features to build datasets, and conducts just-in-time prediction on the defects of the intelligent computing framework at the code submission level. Additionally, LDA is employed to mine codes and code submission information as new features, and then the random forest is adopted for prediction. Results show that the average AUC-ROC is 0.77, and semantic information can slightly improve the prediction performance. Finally, this study leverages an explainable machine learning method called SHAP to analyze the influence of each feature on the prediction output of the model. The findings are as follows. (1) The influence of basic features on the model conforms to traditional software development laws. (2) Code and semantic features in submitted information are important in the prediction result of the model. (3) The contribution of different features in different systems to the output of the prediction model varies a lot. |
| Key words: intelligent computing framework just-in-time defect prediction explainable artificial intelligence empirical software engineering |
