| 摘要: |
| 受限玻尔兹曼机（restricted Boltzmann machine，简称RBM）是一种概率无向图，传统的RBM模型假设隐藏层单元是二值的，二值单元的优势在于计算过程和采样过程相对简单，然而二值化会对基于隐藏层单元的特征提取和数据重构过程带来信息损失.因此，将RBM的可见层单元和隐藏层单元实值化并保持模型训练的有效性，是目前RBM理论研究的重点问题.为了解决这个问题，将二值单元拓展为实值单元，利用实值单元建模数据并提取特征.具体而言，在可见层单元和隐藏层单元之间增加辅助单元，然后将图正则化项引入到能量函数中，基于二值辅助单元和图正则化项，流形上的数据有更高的概率被映射为参数化的截断高斯分布；同时，远离流形的数据有更高的概率被映射为高斯噪声.由此，模型的隐层单元可以被表示为参数化截断高斯分布或高斯噪声的采样实值.该模型称为基于辅助单元的受限玻尔兹曼机（restricted Boltzmann machine with auxiliary units，简称ARBM）.在理论上分析了模型的有效性，然后构建了相应的深度模型，并通过实验验证模型在图像重构任务和图像生成任务中的有效性. |
| 关键词: 受限玻尔兹曼机 神经网络 概率图模型 深度学习 |
| DOI：10.13328/j.cnki.jos.006126 |
| 分类号:TP18 |
| 基金项目:国家自然科学基金（61976216，61672522） |
|
| Deep Generative Neural Networks Based on Real-valued RBM with Auxiliary Hidden Units |
|
ZHANG Jian1,2, DING Shi-Fei1,2, DING Ling3, ZHANG Cheng-Long1
|
|
1.School of Computer Science and Technology, China University of Mining and Technology, Xuzhou 221116, China;2.Engineering Research Center of Mine Digitization of Ministry of Education, Xuzhou 221116, China;3.Xuhai College, China University of Mining and Technology, Xuzhou 221008, China
|
| Abstract: |
| Restricted Boltzmann machine (RBM) is a probabilistic undirected graph, and most traditional RBM models assume that their hidden layer units are binary. The advantage of binary units is their calculation process and sampling process are relatively simple. However, binarized hidden units may bring information loss to the process of feature extraction and data reconstruction. Therefore, a key research point of RBM theory is to construct real-valued visible layer units and hidden layer units, meanwhile, maintain the effectiveness of model training. In this study, the binary units are extended to real-valued units to model data and extract features. To achieve this, specifically, an auxiliary unit is added between the visible layer and the hidden layer, and then the graph regularization term is introduced into the energy function. Based on the binary auxiliary unit and graph regularization term, the data on the manifold has a higher probability to be mapped as a parameterized truncated Gaussian distribution, simultaneously, the data far from the manifold has a higher probability to be mapped as Gaussian noises. The hidden units can be sampled as real-valued units from the parameterized Gaussian distribution and Gaussian noises. In this study, the resulting RBM based model is called restricted Boltzmann machine with auxiliary units (ARBM). Moreover, the effectiveness of the proposed model is analyzed theoretically. The effectiveness of the model in image reconstruction task and image generation task is verified by experiments. |
| Key words: restricted Boltzmann machines neural nets probabilistic undirected graphs deep learning |
