| 引用本文: | 赵祖威,汤恩义,李薛成,戴新宇,陈鑫,李宣东.软件供应链安全中LLM生成代码逻辑性缺陷检测.软件学报,2026,37(7):2871-2885 |
| |
|
| |
|
|
| 本文已被：浏览 1519次 下载 916次 |
码上扫一扫！ |
|
|
| 软件供应链安全中LLM生成代码逻辑性缺陷检测 |
|
赵祖威1,2, 汤恩义1,3, 李薛成1,3, 戴新宇1,2, 陈鑫1,4, 李宣东1,4
|
|
1.计算机软件新技术全国重点实验室 (南京大学), 江苏 南京 210023;2.南京大学 人工智能学院, 江苏 南京 210023;3.南京大学 软件学院, 江苏 南京 210093;4.南京大学 计算机学院, 江苏 南京 210023
|
|
| 摘要: |
| 随着大语言模型(large language model, LLM)在代码生成领域的快速发展, 其生成的代码在智能化基础软件供应链中的应用日益广泛. 基础软件供应链中集成了大量基于LLM生成代码开发的第三方模块与组件. 然而, 由于LLM主要基于开源代码进行训练, 训练代码中的缺陷与安全漏洞可能会导致生成代码存在潜在错误与供应链安全问题. 为此, 学术界有针对性地提出了EvalPlus等测试技术, 但这些技术主要依赖基于概率的测试用例生成机制, 难以实现对供应链关键路径的全面覆盖, 导致深层次逻辑性缺陷难以被有效发现. 为了解决上述问题, 提出一种融合符号执行的供应链LLM生成代码的缺陷检测方法. 该方法通过符号执行挂载机制自动识别LLM生成代码的输入参数并进行适配和符号挂载, 制导符号执行引擎对程序的关键路径进行精确的约束分析, 生成高效的边界测试用例, 从而发现现有方法难以检测到的深层逻辑性程序缺陷. 在现有主流基准数据集上, 对LMSYS Chatbot Arena中排名前11的主流LLM进行了实验评估. 实验结果表明, 该方法能够更有效地检测出LLM生成代码中的逻辑性缺陷, 使代码的平均测试通过率降低了3.99%–18.98%, 平均测试覆盖率提高了3.31%–8.19%, 有效提升了LLM生成代码的正确性和智能化基础软件供应链的安全性. |
| 关键词: 软件缺陷检测 供应链安全 大语言模型 代码生成 符号执行 |
| DOI：10.13328/j.cnki.jos.007588 |
| 分类号:TP311 |
| 基金项目:国家自然科学基金(62172210, 62172211) |
|
| Logical Defect Detection for LLM-generated Code in Software Supply Chain Security |
|
ZHAO Zu-Wei1,2, TANG En-Yi1,3, LI Xue-Cheng1,3, DAI Xin-Yu1,2, CHEN Xin1,4, LI Xuan-Dong1,4
|
|
1.State Key Laboratory for Novel Software Technology (Nanjing University), Nanjing 210023, China;2.School of Artificial Intelligence, Nanjing University, Nanjing 210023, China;3.Software Institute, Nanjing University, Nanjing 210093, China;4.School of Computer Science, Nanjing University, Nanjing 210023, China
|
| Abstract: |
| As the large language models (LLMs) rapidly advance in code generation, their generated code gains increasingly widespread applications in intelligent foundational software supply chains. The foundational software supply chains integrate a large number of third-party modules and components developed by employing LLM-generated code. However, since LLMs are primarily trained based on open-source code, defects and security vulnerabilities in the training code may cause potential errors in the generated code and security problems in the software supply chain. To this end, targeted testing techniques such as EvalPlus have been proposed, but it is difficult for these techniques to achieve the full coverage of critical paths in the supply chain due to their reliance on probability-based test case generation, which makes it hard to uncover deep-seated logical software defects. To solve the above-mentioned problems, this study proposes a defect detection method for LLM-generated code in software supply chains that integrates symbolic execution. This method employs a symbolic execution mounting mechanism to automatically identify input parameters in LLM-generated code and perform adaptation and symbolic mounting. It then guides the symbolic execution engine to conduct precise constraint analysis on the program’s critical paths and generate efficient boundary test cases, thus detecting deep-seated logical software defects that the existing methods struggle to detect. This study conducts experimental evaluation on the top 11 mainstream LLMs from the LMSYS Chatbot Arena by adopting existing mainstream benchmark datasets. Experimental results show that the proposed method can more effectively detect logical defects in LLM-generated code, reducing the average test pass rate by 3.99% to 18.98% and increasing the average test coverage by 3.31% to 8.19%. Finally, the correctness of LLM-generated code and the security of intelligent foundational software supply chains are effectively improved. |
| Key words: software defect detection supply chain security large language model (LLM) code generation symbolic execution |
|
|
|
|
