| 摘要: |
| 建立以受限玻尔兹曼机（restricted Boltzmann machine，简称RBM）为基石的深度网络模型，是深度学习研究的热点领域之一.Point-wise Gated受限玻尔兹曼机（point-wise gated RBM，简称pgRBM）是一种RBM的变种算法.该算法能够在含噪声的数据中自适应地找到数据中与分类有关的部分，从而实现较好的分类结果.假设一组数据中有噪声数据和干净数据，如何应用不含噪声的数据提升pgRBM的性能，是一个重要的研究问题.针对这一问题，首先，在传统的pgRBM基础上提出一种基于随机噪声数据与干净数据的Point-wise Gated受限玻尔兹曼机（pgRBM based on random noisy data and clean data，简称pgrncRBM）方法，其网络中与分类有关权值的初值是通过不含噪声的数据学习得到的，所以pgrncRBM在处理随机噪声数据时可以学习到更为"干净"的数据.在pgrncRBM中，与分类有关的数据与噪声都是使用RBM建模.如果噪声是图片，pgrncRBM就不能很好地去除噪声.Spike-and-Slab RBM（ssRBM）是一种处理实值数据的RBM变种模型，其定义两种不同类型的隐层用来学习实值数据的分布特性.因此，将ssRBM与pgRBM相结合，提出一种基于图像噪声数据与干净数据的Point-wise Gated受限玻尔兹曼机（pgRBM based on image noisy data and clean data，简称pgincRBM）方法.该方法使用ssRBM对噪声建模，其在处理图像噪声数据时可以学习到更为"干净"的数据.然后，通过堆叠pgrncRBM、pgincRBM和传统的RBM构建出深度网络模型，并探讨了权值不确定性方法在提出网络模型中的可行性.最后，在含噪声的手写数据集上进行MATLAB仿真实验.实验结果表明，pgrncRBM和pgincRBM都是有效的神经网络学习方法. |
| 关键词: 受限玻尔兹曼机 深度置信网络 权值不确定性 特征选择 |
| DOI：10.13328/j.cnki.jos.005574 |
| 分类号:TP183 |
| 基金项目:国家自然科学基金（61672522，61379101）；国家重点基础研究发展计划（973）（2013CB329502） |
|
| Deep Belief Network Based on Noisy Data and Clean Data |
|
ZHANG Nan1, DING Shi-Fei1,2,3, ZHANG Jian1, ZHAO Xing-Yu1
|
|
1.School of Computer Science and Technology, China University of Mining and Technology, Xuzhou 221116, China;2.Mine Digitization Engineering Research Center of Ministry of Education(China University of Mining and Technology), Xuzhou 221116, China;3.Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
|
| Abstract: |
| Stacking restricted Boltzmann machines (RBM) to create deep networks, such as deep belief networks (DBN), has become one of the most important research fields in deep learning. Point-wise gated restricted Boltzmann machines (pgRBM), an RBM variant, can effectively find the task-relevant patterns from data containing irrelevant patterns and thus achieves satisfied classification results. Given that train data is composed of noisy data and clean data, how the clean data is applied to promote the performance of the pgRBM is a problem. To address the problem, this study first proposes a method, named as pgRBM based on random noisy data and clean data (pgrncRBM). The pgrncRBM makes use of RBM and the clean data to obtain the initial values of the task-relevant weights, so it can learn the "clean" data from the data containing random noisy. In the pgrncRBM, the general RBM is used to pre-train the weights of task-relevant patterns from data and irrelevant patterns. If the noise is an image, the pgrncRBM cannot learn the task-relevant patterns from the noisy data. Spike-and-Slab RBM, an RBM variant, uses two types of hidden layers to determine the mean and covariance of each visible unit. Threrfore, this study combines ssRBM with pgRBM and proposes a method, named as pgRBM based on image noisy data and clean data (pgincRBM). The pgincRBM uses the ssRBM to model the noise, so it can learn the "clean" data from the data containing image noisy. And then, this study stacks pgrncRBM, pgincRBM, and RBMs to create deep networks, and discusses the feasibility that the weight uncertainty method is developed to prevent overfitting in the proposed networks. Experimental results on MNIST variation datasets show that pgrncRBM and pgincRBM are effective neural networks learning methods. |
| Key words: restricted Boltzmann machine (RBM) deep belief network weight uncertainty feature selection |
