Binary neural networks (BNNs) are highly appealing to the industry due to their significantly reduced computation and storage requirements. However, their accuracy still lags behind that of networks with full-precision parameters. Most existing methods focus on improving the performance of BNNs through advanced training techniques. Empirical findings reveal that the representation capability of quantized features is considerably weaker than that of full-precision features. To address this limitation, a widening and squeezing mechanism is proposed to construct high-accuracy yet compact BNNs. Specifically, features from the original full-precision networks are projected into high-dimensional quantized features to mitigate the representation gap. Meanwhile, redundant quantized features are pruned to prevent the over growth of feature dimensions. As a result, a compact yet sufficiently expressive quantized neural network is constructed. Experimental results on benchmark datasets demonstrate that the proposed method achieves high-accuracy BNNs with significantly fewer parameters and computations while delivering performance comparable to full-precision baseline models. For instance, the binary ResNet-18 achieves a top-1 accuracy of 70% on the ImageNet dataset.