Federated Learning enables collaborative model training among multiple clients without exchanging their local private data. However, this paradigm is vulnerable to increasingly sophisticated poisoning attacks. Existing defense mechanisms often exhibit shortcomings in detection efficiency, generalization to diverse attacks, and performance stability under non-independent and identically distributed (non-IID) data. To address these challenges, this study proposes FedProbe, an interpretable federated poisoning attack defense framework guided by proxy models. FedProbe adopts a two-stage mechanism. In the first stage, under proxy model guidance, the framework computes the KL divergence among local models on a trusted server-side dataset to measure their behavioral similarity and determine the optimal number of clusters. After removing outlier updates, each cluster is aggregated into a proxy model, aiming to effectively partition models trained on different data distributions and improve subsequent detection efficiency. In the second stage, an interpretable analysis is conducted. FedProbe leverages the SHAP technique to perform an in-depth analysis of the proxy models, computing key feature attributions and ultimately using cross-category suspicion scores to identify and remove potentially malicious models accurately. Experimental results demonstrate that FedProbe exhibits superior performance and robustness across various benchmark datasets. In benign settings, the convergence time is only 1.11 to 1.21 times that of standard aggregation algorithms, indicating minimal overhead. In terms of security, even in extreme scenarios with up to 40% malicious clients, FedProbe maintains superior convergence when defending against untargeted attacks and backdoor attacks, while keeping the attack success rate below 25%. Moreover, when facing adaptive attacks, it shows significant advantages in both defense effectiveness and task accuracy.