| 本文已被：浏览 1495次 下载 4684次 |
码上扫一扫！ |
|
|
| 自动化张量分解加速卷积神经网络 |
|
宋冰冰1, 张浩2,3, 吴子锋2,3, 刘俊晖2,3, 梁宇2,3, 周维2,3
|
|
1.云南大学信息学院, 云南 昆明 650504;2.云南大学软件学院, 云南 昆明 650504;3.云南大学跨境网络空间安全工程研究中心, 云南 昆明 650504
|
|
| 摘要: |
| 近年来，卷积神经网络（CNN）展现了强大的性能，被广泛应用到了众多领域.由于CNN参数数量庞大，且存储和计算能力需求高，其难以部署在资源受限设备上.因此，对CNN的压缩和加速成为一个迫切需要解决的问题.随着自动化机器学习（AutoML）的研究与发展，AutoML对神经网络发展产生了深远的影响.受此启发，提出了基于参数估计和基于遗传算法的两种自动化加速卷积神经网络算法.该算法能够在给定精度损失范围内自动计算出最优的CNN加速模型，有效地解决了张量分解中，人工选择秩带来的误差问题，能够有效地提升CNN的压缩和加速效果.通过在MNIST和CIFAR-10数据集上的严格测试，与原网络相比，在MNIST数据集上准确率稍微下降了0.35%，模型的运行时间获得了4.1倍的大幅提升；在CIFAR-10数据集上，准确率稍微下降了5.13%，模型的运行时间获得了0.8倍的大幅提升. |
| 关键词: 张量分解 卷积神经网络 自动化机器学习 神经网络压缩 神经网络加速 |
| DOI：10.13328/j.cnki.jos.006057 |
| 分类号:TP183 |
| 基金项目:国家自然科学基金（61762089，61863036，61663047） |
|
| Automated Tensor Decomposition to Accelerate Convolutional Neural Networks |
|
SONG Bing-Bing1, ZHANG Hao2,3, WU Zi-Feng2,3, LIU Jun-Hui2,3, LIANG Yu2,3, ZHOU Wei2,3
|
|
1.School of Information Science and Engineering, Yunnan University, Kunming 650504, China;2.National Pilot School of Software, Yunnan University, Kunming 650504, China;3.Engineering Research Center of Cyberspace, Yunnan University, Kunming 650504, China
|
| Abstract: |
| Recently, convolutional neural network (CNN) have demonstrated strong performance and are widely used in many fields. Due to the large number of CNN parameters and high storage and computing power requirements, it is difficult to deploy on resource-constrained devices. Therefore, compression and acceleration of CNN models have become an urgent problem to be solved. With the research and development of automatic machine learning (AutoML), AutoML has profoundly impacted the development of neural networks. Inspired by this, this study proposes two automated accelerated CNN algorithms based on parameter estimation and genetic algorithms, which can calculate the optimal accelerated CNN model within a given accuracy loss range, effectively solving the error caused by artificially selected rank in tensor decomposition. It can effectively improve the compression and acceleration effects of the convolutional neural network. By rigorous testing on the MNIST and CIFAR-10 data sets, the accuracy rate on the MNIST dataset is slightly reduced by 0.35% compared to the original network, and the running time of the model is greatly reduced by 4.1 times, the accuracy rate on the CIFAR-10 dataset dropped slightly by 5.13%, and the running time of the model was greatly decreased by 0.8 times. |
| Key words: tensor decomposition convolutional neural network automatic machine learning neural network compression neural network acceleration |
