A strategy is proposed for facial expression recognition under the graph embedding (GE) framework. The neighborhood weighted graph based on the expression similarity is constructed to learn the sub-space. In the sub-space, the data distribute on the manifold based on expression semantic. The proposed sub-space method can overcome the difficulties for facial expression recognition caused by the differences in individuals, lightings, poses. The expressions of the facial images in the data set are exploited in a semi-supervised way. Expression similarity between two facial images is measured by the dot product of the expression fuzzy membership function vectors. Experimental results on Cohn-Kanade and the data set of this paper demonstrate the effectiveness of the approach.