Logical Defect Detection for LLM-generated Code in Software Supply Chain Security
Author:
Affiliation:

Clc Number:

TP311

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    As the large language models (LLMs) rapidly advance in code generation, their generated code gains increasingly widespread applications in intelligent foundational software supply chains. The foundational software supply chains integrate a large number of third-party modules and components developed by employing LLM-generated code. However, since LLMs are primarily trained based on open-source code, defects and security vulnerabilities in the training code may cause potential errors in the generated code and security problems in the software supply chain. To this end, targeted testing techniques such as EvalPlus have been proposed, but it is difficult for these techniques to achieve the full coverage of critical paths in the supply chain due to their reliance on probability-based test case generation, which makes it hard to uncover deep-seated logical software defects. To solve the above-mentioned problems, this study proposes a defect detection method for LLM-generated code in software supply chains that integrates symbolic execution. This method employs a symbolic execution mounting mechanism to automatically identify input parameters in LLM-generated code and perform adaptation and symbolic mounting. It then guides the symbolic execution engine to conduct precise constraint analysis on the program’s critical paths and generate efficient boundary test cases, thus detecting deep-seated logical software defects that the existing methods struggle to detect. This study conducts experimental evaluation on the top 11 mainstream LLMs from the LMSYS Chatbot Arena by adopting existing mainstream benchmark datasets. Experimental results show that the proposed method can more effectively detect logical defects in LLM-generated code, reducing the average test pass rate by 3.99% to 18.98% and increasing the average test coverage by 3.31% to 8.19%. Finally, the correctness of LLM-generated code and the security of intelligent foundational software supply chains are effectively improved.

    Reference
    Related
    Cited by
Get Citation

赵祖威,汤恩义,李薛成,戴新宇,陈鑫,李宣东.软件供应链安全中LLM生成代码逻辑性缺陷检测.软件学报,2026,37(7):2871-2885

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:September 08,2025
  • Revised:October 20,2025
  • Adopted:
  • Online: December 26,2025
  • Published: July 06,2026
You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063