| 本文已被：浏览 1245次 下载 3142次 |
码上扫一扫！ |
|
|
| 基于平行多尺度时空图卷积网络的三维人体姿态估计算法 |
|
杨红红1,2, 刘泓希1, 张玉梅2,3, 吴晓军1,2,3
|
|
1.现代教学技术教育部重点实验室(陕西师范大学), 陕西 西安 710062;2.民歌智能计算与服务技术文化和旅游部重点实验室(陕西师范大学), 陕西 西安 710062;3.陕西师范大学 计算机科学学院, 陕西 西安 710062
|
|
| 摘要: |
| 针对基于图卷积神经网络(GCN)的人体姿态估计方法不能充分聚合关节点时空特征、限制判别性特征提取的问题, 构造基于平行多尺度时空图卷积的网络模型(PMST-GNet), 提高三维人体姿态估计的性能. 该模型首先设计对角占优的时空注意力图卷积(DDA-STGConv), 构建跨域时空邻接矩阵, 对骨架关节点信息进行基于自约束和注意力机制约束的建模, 增强节点间的信息交互; 然后, 通过设计图拓扑聚合函数构造不同的图拓扑结构, 以DDA-STGConv为基本单元构建平行多尺度子网络模块(PM-SubGNet); 最后, 为了更好地提取骨架关节的上下文信息, 设计多尺度特征交叉融合模块(MFEB), 实现平行子图网络之间多尺度信息的交互, 提高GCN的特征表示能力. 在主流3D姿态估计数据集Human3.6M和MPI-INF-3DHP数据集上的对比实验结果表明, 所提PMST-GNet模型在三维人体姿态估计中具有较好的效果, 优于Sem-GCN、GraphSH、UGCN等当前基于GCN网络的主流算法. |
| 关键词: 三维人体姿态估计 对角占优的时空注意力图卷积 平行多尺度子网络 多尺度特征交叉融合 |
| DOI：10.13328/j.cnki.jos.007200 |
| 分类号:TP391 |
| 基金项目:国家自然科学基金(61907028, 11872036); 陕西省青年科技新星项目(2021KJXX-91); 文化和旅游部重点实验室资助项目(2023-02, 2022-13); 陕西省自然科学基金面上项目(2024JC-YBMS-503) |
|
| Parallel Multi-scale Spatio-temporal Graph Convolutional Network for 3D Human Pose Estimation |
|
YANG Hong-Hong1,2, LIU Hong-Xi1, ZHANG Yu-Mei2,3, WU Xiao-Jun1,2,3
|
|
1.Key Laboratory of Modern Teaching Technology (Shaanxi Normal University), Ministry of Education, Xi’an 710062, China;2.Key Laboratory of Intelligent Computing and Service Technology for Folk Song (Shaanxi Normal University), Ministry of Culture and Tourism, Xi’an 710062, China;3.School of Computer Science, Shaanxi Normal University, Xi’an 710062, China
|
| Abstract: |
| As the human pose estimation (HPE) method based on graph convolutional network (GCN) cannot sufficiently aggregate spatiotemporal features of skeleton joints and restrict discriminative features extraction, in this paper, a parallel multi-scale spatio-temporal graph convolutional network (PMST-GNet) model is built to improve the performance of 3D HPE. Firstly, a diagonally dominant spatiotemporal attention graph convolutional layer (DDA-STGConv) is designed to construct a cross-domain spatiotemporal adjacency matrix and model the joint features based on self-constraint and attention mechanism constrain, therefore enhancing information interaction among nodes. Then, a graph topology aggregation function is devised to construct different graph topologies, and a parallel multi-scale sub-graph network module (PM-SubGNet) is constructed with DDA-STGConv as the basic unit. Finally, a multi-scale feature cross fusion block (MFEB) is designed, by which multi-scale information among PM-SubGNets can interact to improve the feature representation of GCN, therefore better extracting the context information of skeleton joints. The experimental results on the mainstream 3D HPE datasets Human3.6M and MPI-INF-3DHP show that the proposed PMST-GNet model has a good effect in 3D HPE and is superior to the current mainstream GCN-based algorithms such as Sem-GCN, GraphSH, and UGCN. |
| Key words: 3D human pose estimation (3D HPE) diagonally dominant spatio-temporal attention graph convolution parallel multi-scale sub-graph network multi-scale feature cross fusion |
