| 摘要: |
| 利用Kinect相机结合增强现实技术和手势识别方法设计并实现了一个弓弦乐器虚拟演奏系统——以二胡为例.将Kinect获取的现实场景和虚拟乐器融合在一起绘制成增强现实场景.通过Kinect得到的深度数据和贝叶斯肤色模型将用户的左手分割出来,并再次绘制在增强图像上形成新的图像,从而解决虚拟演奏场景中的虚实遮挡问题.利用基于反向动力学和马尔可夫模型的三维虚拟手势拟合方法,对演奏过程中的左手手势进行识别,并结合右手的运动状态完成乐器的虚拟演奏. |
| 关键词: 虚拟演奏 Kinect 增强现实 手势识别 |
| DOI： |
| 分类号: |
| 基金项目:国家社会科学基金(12AZD120);国家自然科学基金(61332017, 61173124);浙江省自然科学基金(LY14F020037);国家科技支撑计划(2015BAK04B00) |
|
| Kinect-Based Virtual Performance System of National Bowed String Instruments |
|
ZHANG Shi-Cheng1, CHEN Gen-Fang1, ZHANG Ming-Min2, ZHAO Pei-Tao2, PAN Zhi-Geng1
|
|
1.Digital Media & Interaction Research Center, Hangzhou Normal University, Hangzhou 311121, China;2.State Key Laboratory of CAD & CG(Zhejiang University), Hangzhou 310058, China
|
| Abstract: |
| Using Kinect camera combined with augmented reality and hand gesture recognition method, this paper designs and implements a virtual playing system for bowstring musical instruments (e.g. erhu). The fusion rendering of real scene captured by Kinect and virtual instrument forms the augmented reality scene. Through the depth data and a Bayesian skin model, the system segments the user's left hand region, then draws the hand image on the enhanced image, thus solving the problem of occlusion between virtual and real worlds in augmented reality system. Based on inverse kinematic methods and Markov model the presented design also provides a 3D virtual gesture fitting method to recognize left hand gestures, then completes the virtual instruments playing combined with right hand movement status. |
| Key words: virtual performance Kinect augmented reality gesture recognition |
