Journal of Software:2018.29(S2):30-43

(国家广播电视总局 广播科学研究院 信息技术研究所, 北京 100866;北京工商大学 计算机与信息工程学院, 北京 100048)
Image Description Method Based on Generative Adversarial Networks
XUE Zi-Yu,GUO Pei-Yu,Zhu Xiao-Bin,ZHANG Nai-Guang
(Information Technology Institute, Academy of Broadcasting Science, National Radio and Television Administration, Beijing 100866, China;School of Computer and Information Engineering, Beijing Technology and Business University, Beijing 100048, China)
Chart / table
Similar Articles
Article :Browse 593   Download 935
Received:April 16, 2018    
> 中文摘要: 近年来,深度学习在图像描述领域得到越来越多的关注.现有的深度模型方法一般通过卷积神经网络进行特征提取,递归神经网络对特征拼接生成语句.然而,当图像较为复杂时,特征提取不准确且语句生成模型模式固定,部分语句不具备连贯性.基于此,提出一种结合多频道特征提取模型与生成式对抗网络框架的图像描述方法——CACNN-GAN.此方法在卷积层加入频道注意力机制在各频道提取特征,与COCO图像集进行近似特征比对,选择排序靠前的图像特征作为生成式对抗网络的输入,通过生成器与鉴别器之间的博弈过程,训练句法多样、语句通顺、词汇丰富的语句生成器模型.在实际数据集上的实验结果表明,CACNN-GAN能够有效地对图像进行语义描述,相比其他主流算法,显示出了更高的准确率.
Abstract:In recent years, deep learning has gained more and more attention in image description. The existing deep learning methods using CNNs to extract features and RNNs to fold into one sentence. Nevertheless, when dealing with complex images, the feature extraction is inaccurate. And the fixed mode of sentence generation model leads to inconsistent sentences. To solve this problem, this study proposes a method combine channel-wise attention model and GANs, named CACNN-GAN. The channel-wise attention mechanism is added to each conv-layer to extract features, compare with the COCO dataset, and select the top features to generate sentence. Using GANs to generate the sentences, which is generated by the game process between the generator and the discriminator. After that, we can get a sentence generator contains the varied syntax, smooth sentence, and rich vocabulary. Experiments on real datasets illustrates that CACNN-GAN can effectively describe images, and get higher accuracy compared with the state-of-art.
文章编号:     中图分类号:    文献标志码:
基金项目:国家广播电视总局广播科学研究院基本科研业务费课题(130016018000123) 国家广播电视总局广播科学研究院基本科研业务费课题(130016018000123)
Foundation items:Basal Research Fund of Academy of Broadcasting Science, National Radio and Television Administration (130016018000123)
Reference text:


XUE Zi-Yu,GUO Pei-Yu,Zhu Xiao-Bin,ZHANG Nai-Guang.Image Description Method Based on Generative Adversarial Networks.Journal of Software,2018,29(S2):30-43