Since its inception, the Java programming language has been in a process of constant development and evolution. As new language features and programming paradigms continuously emerge, Java’s expressiveness and execution efficiency are steadily improving, driving progress in the whole software ecosystem. To ensure the security and stability of the Java ecosystem, researchers have designed multiple test program generation methods for Java compilers and virtual machines to detect potential defects. However, the existing research primarily focuses on designing test program generation methods or mutation strategies for mature Java syntax, making it challenging to effectively test new language features. To this end, this study proposes LumiX, a test program generation method for Java’s new features based on large language models (LLMs). First, LumiX adopts LLMs to summarize new language features using natural language descriptions and obtains the usage descriptions of the new features. Then, it employs historically bug-revealing test programs as seed programs and applies program analysis tools to extract reusable program elements from the seed programs. By combining the usage descriptions of new features, LLMs are employed to generate code snippets that incorporate the new features. The newly generated code snippets are then inserted into the seed programs to generate test programs that can cover the new features. Finally, LumiX designs a dual-layer differential testing method, which utilizes different Java compilers (javac and ECJ) and virtual machines (HotSpot and OpenJ9) to compile and execute the newly generated test programs, with potential defects detected by comparing the execution results of different compilers and virtual machines. Experimental results show that LumiX can effectively generate test programs that cover Java’s new language features and enhance the testing capabilities of existing tools for Java’s new language features. Additionally, this study applies LumiX to the latest released Java compilers and virtual machines, and a total of 16 unknown defects have been discovered, 12 of which have been confirmed or fixed by developers.