Although deep neural networks (DNNs) have been widely used in various fields, they exhibit significant vulnerability in adversarial environments. In recent years, a specialized form of attack has emerged, called backdoor attacks or Trojan attacks. Attackers intentionally implant backdoors in DNNs, causing backdoored models to make correct predictions on normal inputs but produce incorrect predictions on inputs containing triggers. To address the threat of backdoor attacks, researchers have proposed various detection methods. However, these methods rely on strict assumptions, such as white-box access to the target model or knowledge of trigger patterns, and require a large amount of detection data and detection time, which limits their applicability in real-world scenarios. Moreover, existing methods have insufficient research on identifying target classes in backdoor attacks and fail to effectively reveal the critical role of target classes in attacks and their impact on real-world tasks. This study proposes an efficient black-box backdoor detection method (EBLD). With only a small amount of clean data, transfer learning is leveraged to train a small set of benign models and backdoored models. Subsequently, a large number of models are rapidly generated through model mutation techniques. Finally, based on these models, multiple classifiers are trained to jointly determine whether the target model is a backdoored model. In the absence of detection data, this study proposes a “feature adaptation” strategy, which leverages benign models and backdoored models generated on other tasks to train binary classifiers capable of detecting new target models. This strategy overcomes the reliance on detection samples and fully utilizes existing benign and backdoored model resources, thus improving detection efficiency. In addition, a binary classifier fine-tuning method based on transfer learning is designed. By loading the weights of existing binary classifiers to train new binary classifiers, backdoor detection time can be further reduced. Finally, an optimal query set library is constructed based on the optimal query sets obtained during binary classifier training. By analyzing the output class distribution of backdoored models on the optimal query sets, the target class of the backdoor attack is quickly identified. Target class identification not only helps to more precisely locate the attack scope but also provides clear guidance for backdoor defense and repair. Experimental results demonstrate that the proposed method significantly outperforms existing methods in detection accuracy and achieves a high level of detection efficiency.