| 摘要: |
| 二值神经网络(binary neural network, BNN)因其较少的计算和存储开销而对业界非常有吸引力, 但其准确率仍然比全精度参数的网络差. 大多数现有方法旨在通过利用更有效的训练技术来提高二值神经网络的性能. 然而, 通过实验发现量化后特征的表示能力远弱于全精度的特征. 因此, 提出一种加宽和收缩机制来构建高精度而紧凑的二值神经网络. 首先, 通过将原始全精度网络中的特征投影到高维量化特征来解决量化特征表示能力弱的问题. 同时, 冗余的量化特征将被消除, 以避免某些特征维度的过度增长. 进而建立一个紧凑但具有足够表示能力的量化神经网络. 基准数据集上的实验结果表明, 该方法能够以更少的参数量和计算量建立高精度二值神经网络, 其准确率与全精度基线模型几乎相同, 例如, 二值量化的ResNet-18 在ImageNet数据集上达到了70%的准确率. |
| 关键词: 神经网络 模型量化 图像分类 目标检测 |
| DOI：10.13328/j.cnki.jos.007363 |
| 分类号:TP301 |
| 基金项目:国家自然科学基金(62332015) |
|
| Widening and Squeezing Mechanism for Binary Neural Network |
|
HAN Kai1,2, LIU Chuan-Jian3, WU En-Hua1,4
|
|
1.State Key Laboratory of Computer Science (Institute of Software, Chinese Academy of Sciences), Beijing 100190, China;2.University of Chinese Academy of Sciences, Beijing 100049, China;3.Huawei Noah’s Ark Lab, Beijing 100085, China;4.Faculty of Science and Technology, University of Macau, Macao 999078, China
|
| Abstract: |
| Binary neural networks (BNNs) are highly appealing to the industry due to their significantly reduced computation and storage requirements. However, their accuracy still lags behind that of networks with full-precision parameters. Most existing methods focus on improving the performance of BNNs through advanced training techniques. Empirical findings reveal that the representation capability of quantized features is considerably weaker than that of full-precision features. To address this limitation, a widening and squeezing mechanism is proposed to construct high-accuracy yet compact BNNs. Specifically, features from the original full-precision networks are projected into high-dimensional quantized features to mitigate the representation gap. Meanwhile, redundant quantized features are pruned to prevent the over growth of feature dimensions. As a result, a compact yet sufficiently expressive quantized neural network is constructed. Experimental results on benchmark datasets demonstrate that the proposed method achieves high-accuracy BNNs with significantly fewer parameters and computations while delivering performance comparable to full-precision baseline models. For instance, the binary ResNet-18 achieves a top-1 accuracy of 70% on the ImageNet dataset. |
| Key words: neural network model quantization image classification object detection |
