###
Journal of Software:2019.30(11):3326-3339

基于噪声数据与干净数据的深度置信网络
张楠,丁世飞,张健,赵星宇
(中国矿业大学 计算机科学与技术学院, 江苏 徐州 221116;中国矿业大学 计算机科学与技术学院, 江苏 徐州 221116;矿山数字化教育部工程研究中心(中国矿业大学), 江苏 徐州 221116;中国科学院 计算技术研究所 智能信息处理重点实验室, 北京 100190)
Deep Belief Network Based on Noisy Data and Clean Data
ZHANG Nan,DING Shi-Fei,ZHANG Jian,ZHAO Xing-Yu
(School of Computer Science and Technology, China University of Mining and Technology, Xuzhou 221116, China;School of Computer Science and Technology, China University of Mining and Technology, Xuzhou 221116, China;Mine Digitization Engineering Research Center of Ministry of Education(China University of Mining and Technology), Xuzhou 221116, China;Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China)
Abstract
Chart / table
Reference
Similar Articles
Article :Browse 60   Download 52
Received:October 23, 2017    Revised:December 25, 2017
> 中文摘要: 建立以受限玻尔兹曼机(restricted Boltzmann machine,简称RBM)为基石的深度网络模型,是深度学习研究的热点领域之一.Point-wise Gated受限玻尔兹曼机(point-wise gated RBM,简称pgRBM)是一种RBM的变种算法.该算法能够在含噪声的数据中自适应地找到数据中与分类有关的部分,从而实现较好的分类结果.假设一组数据中有噪声数据和干净数据,如何应用不含噪声的数据提升pgRBM的性能,是一个重要的研究问题.针对这一问题,首先,在传统的pgRBM基础上提出一种基于随机噪声数据与干净数据的Point-wise Gated受限玻尔兹曼机(pgRBM based on random noisy data and clean data,简称pgrncRBM)方法,其网络中与分类有关权值的初值是通过不含噪声的数据学习得到的,所以pgrncRBM在处理随机噪声数据时可以学习到更为"干净"的数据.在pgrncRBM中,与分类有关的数据与噪声都是使用RBM建模.如果噪声是图片,pgrncRBM就不能很好地去除噪声.Spike-and-Slab RBM(ssRBM)是一种处理实值数据的RBM变种模型,其定义两种不同类型的隐层用来学习实值数据的分布特性.因此,将ssRBM与pgRBM相结合,提出一种基于图像噪声数据与干净数据的Point-wise Gated受限玻尔兹曼机(pgRBM based on image noisy data and clean data,简称pgincRBM)方法.该方法使用ssRBM对噪声建模,其在处理图像噪声数据时可以学习到更为"干净"的数据.然后,通过堆叠pgrncRBM、pgincRBM和传统的RBM构建出深度网络模型,并探讨了权值不确定性方法在提出网络模型中的可行性.最后,在含噪声的手写数据集上进行MATLAB仿真实验.实验结果表明,pgrncRBM和pgincRBM都是有效的神经网络学习方法.
Abstract:Stacking restricted Boltzmann machines (RBM) to create deep networks, such as deep belief networks (DBN), has become one of the most important research fields in deep learning. Point-wise gated restricted Boltzmann machines (pgRBM), an RBM variant, can effectively find the task-relevant patterns from data containing irrelevant patterns and thus achieves satisfied classification results. Given that train data is composed of noisy data and clean data, how the clean data is applied to promote the performance of the pgRBM is a problem. To address the problem, this study first proposes a method, named as pgRBM based on random noisy data and clean data (pgrncRBM). The pgrncRBM makes use of RBM and the clean data to obtain the initial values of the task-relevant weights, so it can learn the "clean" data from the data containing random noisy. In the pgrncRBM, the general RBM is used to pre-train the weights of task-relevant patterns from data and irrelevant patterns. If the noise is an image, the pgrncRBM cannot learn the task-relevant patterns from the noisy data. Spike-and-Slab RBM, an RBM variant, uses two types of hidden layers to determine the mean and covariance of each visible unit. Threrfore, this study combines ssRBM with pgRBM and proposes a method, named as pgRBM based on image noisy data and clean data (pgincRBM). The pgincRBM uses the ssRBM to model the noise, so it can learn the "clean" data from the data containing image noisy. And then, this study stacks pgrncRBM, pgincRBM, and RBMs to create deep networks, and discusses the feasibility that the weight uncertainty method is developed to prevent overfitting in the proposed networks. Experimental results on MNIST variation datasets show that pgrncRBM and pgincRBM are effective neural networks learning methods.
文章编号:     中图分类号:TP183    文献标志码:
基金项目:国家自然科学基金(61672522,61379101);国家重点基础研究发展计划(973)(2013CB329502) 国家自然科学基金(61672522,61379101);国家重点基础研究发展计划(973)(2013CB329502)
Foundation items:National Natural Science Foundation of China (61672522, 61379101); National Program on Key Basic Research Project of China (973) (2013CB329502)
Reference text:

张楠,丁世飞,张健,赵星宇.基于噪声数据与干净数据的深度置信网络.软件学报,2019,30(11):3326-3339

ZHANG Nan,DING Shi-Fei,ZHANG Jian,ZHAO Xing-Yu.Deep Belief Network Based on Noisy Data and Clean Data.Journal of Software,2019,30(11):3326-3339