| 引用本文: | 李信金,王文婕,王凯,脱厚珍,王诗雅,孙伟,谭小慧,田丰.视听协同的交互式步态干预训练.软件学报,2026,37(5):2006-2023 |
| |
|
| |
|
|
| 本文已被：浏览 2007次 下载 1191次 |
码上扫一扫！ |
|
|
| 视听协同的交互式步态干预训练 |
|
李信金1,2, 王文婕1,2, 王凯3, 脱厚珍4, 王诗雅4, 孙伟1, 谭小慧3, 田丰1
|
|
1.中国科学院 软件研究所, 北京 100190;2.中国科学院大学 计算机科学与技术学院, 北京 101408;3.首都师范大学 信息工程学院, 北京 100089;4.首都医科大学附属北京友谊医院 神经内科, 北京 100050
|
|
| 摘要: |
| 帕金森病(Parkinson’s disease, PD)影响着全球近千万的患者, 尚无根治方法, 但循证医学表明基于感官信息提示的训练可以延缓疾病进展. 然而, 目前的研究大多基于单一模态, 且缺乏对于用户的感知与反馈. 为此, 提出了视听协同的多模态步态训练方法, 基于用户步态数据生成并动态调节多模态提示, 进而探究其辅助早期PD康复的可行性. 该方法首先构建了多模态提示生成框架, 通过用户步态数据计算周期和步高参数, 生成视觉与听觉协同的提示; 然后搭建了交互式干预训练系统, 基于用户步态变化动态调整视听提示, 实现了用户感知与多模态提示生成的交互式迭代. 最后, 在临床招募了40名早期PD患者(H&Y分期≤2)进行实验, 与对照组相比视听协同组改善效果最优, 与基线状态相比视听协同组在训练中和训练后步态对称性平均提高20.776% (p=0.0001)和21.157% (p=0.0001), 病患侧步速平均提高33.924% (p=0.0001)和36.433% (p<0.0001). 结果同时表明, 视听协同提示能够更快速、更持久地帮助患者改善步态表现. 所提出的基于步态数据生成多模态提示的训练方法, 为建立量化驱动的精准康复模式提供了新思路, 促进了多模态交互技术在医疗领域的应用与发展. |
| 关键词: 多模态人机交互 步态感知 多模态提示生成 自适应交互式干预 |
| DOI：10.13328/j.cnki.jos.007540 |
| 分类号:TP181 |
| 基金项目:国家重点研发计划 (2024YFB2808801); 国家自然科学基金(62332003, 82371254); 中国科学院软件研究所重大项目(ISCAS-ZD-202401) |
|
| Interactive Gait Intervention Training with Audiovisual Synergy |
|
LI Xin-Jin1,2, WANG Wen-Jie1,2, WANG Kai3, TUO Hou-Zhen4, WANG Shi-Ya4, SUN Wei1, TAN Xiao-Hui3, TIAN Feng1
|
|
1.Institute of Software, Chinese Academy of Sciences, Beijing 100190, China;2.School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 101408, China;3.Information Engineering College, Capital Normal University, Beijing 100089, China;4.Department of Neurology, Beijing Friendship Hospital, Capital Medical University, Beijing 100050, China
|
| Abstract: |
| Parkinson’s disease (PD) affects nearly 10 million people worldwide, and there is no cure, but evidence-based medicine suggests that training based on sensory cues can slow disease progression. However, most current studies are based on a single modality and lack user perception and feedback. This study proposes an audiovisual collaborative multimodal gait training method, which generates and dynamically adjusts multimodal cues based on users’ gait data to investigate the feasibility of assisting early-stage PD rehabilitation. The method constructs a multimodal cue generation framework to generate visual and auditory cues by calculating cycle and step height parameters from gait data. Then, an interactive intervention training system is built to dynamically adjust the audiovisual cues based on gait changes, which realizes the interactive iteration between user perception and multimodal cue generation. Finally, 40 patients with early-stage PD (H&Y stage≤2) are recruited for a clinical experiment. Compared with the control group, the audiovisual synergy group shows the best improvement effect. Compared with baseline, the gait symmetry in the audiovisual synergy group increases by an average of 20.776% (p=0.0001) during the training and 21.157% (p=0.0001) after the training, and the velocity on the affected side increases by an average of 33.924% (p=0.0001) during the training and 36.433% (p<0.0001) after the training. The results indicate that audiovisual synergistic cues can help patients improve gait performance more quickly and sustainably. The proposed multimodal cueing training method based on gait data provides a new approach for the establishment of a quantitatively driven precision rehabilitation model, and promotes the application and development of multimodal interaction technology in the medical field. |
| Key words: multimodal human-computer interaction gait perception multimodal cue generation adaptive interactive intervention |
|
|
|
|
