This paper proposes a feature extraction algorithm based on the principal component analysis of the anisotropic Gaussian kernel penalty which is different from the traditional kernel principal component analysis algorithms. In the non-linear data dimensionality reduction, the infinite steel tempering of raw data is ignored by the traditional kernel principal component analysis algorithms. Meanwhile, the previous kernel function is mainly controlled by one identical kernel width parameter in each dimension, which cannot reflect the significance of different features in each dimension precisely, resulting the low accuracy of dimensionality reduction process. To address the above issues, contraposing the current problem of infinite steel tempering of raw data, an averaging algorithm is presented in this paper, which has shown good performance in improving the variance contribution rate of the original data typically. Then, anisotropic Gaussian kernel function is introduced owing each dimension has different kernel width parameters which can critically reflect the importance of the dimension data features. In addition, the feature penalty function of kernel principal component analysis is formulated based on the anisotropic Gaussian kernel function to represent the raw data with fewer features and reflect the importance of each principal component information. Furthermore, the gradient descent method is introduced to update the kernel width of feature penalty function and control the iterative process of the feature extraction algorithm. To verify the effectiveness of the proposed algorithm, several algorithms are compared on UCI public data sets and KDDCUP99 data sets respectively. The experimental results show that the feature extraction algorithm of the principal component analysis based on the anisotropic Gaussian kernel penalty is 4.49% higher on average than the previous principal component analysis algorithms on UCI public data sets. The feature extraction algorithm of the principal component analysis based on the anisotropic Gaussian kernel penalty is 8% higher on average than the previous principal component analysis algorithms on KDDCUP99 data sets.