基于实值RBM的深度生成网络研究
作者:
作者单位:

作者简介:

张健(1990-),男,博士,讲师,CCF专业会员,主要研究领域为机器学习,模式识别.
丁世飞(1963-),男,博士,教授,博士生导师,CCF杰出会员,主要研究领域为智能信息处理,人工智能与模式识别,机器学习与数据挖掘,粗糙集与软计算,大数据分析与云计算.
丁玲(1994-),女,讲师,主要研究领域为机器学习,数据挖掘.
张成龙(1992-),男,博士生,主要研究领域为机器学习,模式识别.

通讯作者:

丁世飞,E-mail:dingsf@cumt.edu.cn;张健,E-mail:597409675@qq.com

中图分类号:

TP18

基金项目:

国家自然科学基金(61976216,61672522)


Deep Generative Neural Networks Based on Real-valued RBM with Auxiliary Hidden Units
Author:
Affiliation:

Fund Project:

National Natural Science Foundation of China (61976216, 61672522)

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    受限玻尔兹曼机(restricted Boltzmann machine,简称RBM)是一种概率无向图,传统的RBM模型假设隐藏层单元是二值的,二值单元的优势在于计算过程和采样过程相对简单,然而二值化会对基于隐藏层单元的特征提取和数据重构过程带来信息损失.因此,将RBM的可见层单元和隐藏层单元实值化并保持模型训练的有效性,是目前RBM理论研究的重点问题.为了解决这个问题,将二值单元拓展为实值单元,利用实值单元建模数据并提取特征.具体而言,在可见层单元和隐藏层单元之间增加辅助单元,然后将图正则化项引入到能量函数中,基于二值辅助单元和图正则化项,流形上的数据有更高的概率被映射为参数化的截断高斯分布;同时,远离流形的数据有更高的概率被映射为高斯噪声.由此,模型的隐层单元可以被表示为参数化截断高斯分布或高斯噪声的采样实值.该模型称为基于辅助单元的受限玻尔兹曼机(restricted Boltzmann machine with auxiliary units,简称ARBM).在理论上分析了模型的有效性,然后构建了相应的深度模型,并通过实验验证模型在图像重构任务和图像生成任务中的有效性.

    Abstract:

    Restricted Boltzmann machine (RBM) is a probabilistic undirected graph, and most traditional RBM models assume that their hidden layer units are binary. The advantage of binary units is their calculation process and sampling process are relatively simple. However, binarized hidden units may bring information loss to the process of feature extraction and data reconstruction. Therefore, a key research point of RBM theory is to construct real-valued visible layer units and hidden layer units, meanwhile, maintain the effectiveness of model training. In this study, the binary units are extended to real-valued units to model data and extract features. To achieve this, specifically, an auxiliary unit is added between the visible layer and the hidden layer, and then the graph regularization term is introduced into the energy function. Based on the binary auxiliary unit and graph regularization term, the data on the manifold has a higher probability to be mapped as a parameterized truncated Gaussian distribution, simultaneously, the data far from the manifold has a higher probability to be mapped as Gaussian noises. The hidden units can be sampled as real-valued units from the parameterized Gaussian distribution and Gaussian noises. In this study, the resulting RBM based model is called restricted Boltzmann machine with auxiliary units (ARBM). Moreover, the effectiveness of the proposed model is analyzed theoretically. The effectiveness of the model in image reconstruction task and image generation task is verified by experiments.

    参考文献
    相似文献
    引证文献
引用本文

张健,丁世飞,丁玲,张成龙.基于实值RBM的深度生成网络研究.软件学报,2021,32(12):3802-3813

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2020-04-14
  • 最后修改日期:2020-06-05
  • 录用日期:
  • 在线发布日期: 2021-12-02
  • 出版日期:
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号