Abstract:As the large language models (LLMs) rapidly advance in code generation, their generated code gains increasingly widespread applications in intelligent foundational software supply chains. The foundational software supply chains integrate a large number of third-party modules and components developed by employing LLM-generated code. However, since LLMs are primarily trained based on open-source code, defects and security vulnerabilities in the training code may cause potential errors in the generated code and security problems in the software supply chain. To this end, targeted testing techniques such as EvalPlus have been proposed, but it is difficult for these techniques to achieve the full coverage of critical paths in the supply chain due to their reliance on probability-based test case generation, which makes it hard to uncover deep-seated logical software defects. To solve the above-mentioned problems, this study proposes a defect detection method for LLM-generated code in software supply chains that integrates symbolic execution. This method employs a symbolic execution mounting mechanism to automatically identify input parameters in LLM-generated code and perform adaptation and symbolic mounting. It then guides the symbolic execution engine to conduct precise constraint analysis on the program’s critical paths and generate efficient boundary test cases, thus detecting deep-seated logical software defects that the existing methods struggle to detect. This study conducts experimental evaluation on the top 11 mainstream LLMs from the LMSYS Chatbot Arena by adopting existing mainstream benchmark datasets. Experimental results show that the proposed method can more effectively detect logical defects in LLM-generated code, reducing the average test pass rate by 3.99% to 18.98% and increasing the average test coverage by 3.31% to 8.19%. Finally, the correctness of LLM-generated code and the security of intelligent foundational software supply chains are effectively improved.