基于视觉显著度的说话检测

基于视觉显著度的说话检测 Visual Voice Activity Detection Based on Saliency Detection 王瑾 WANG Jin 1 first-author 梁超 LIANG Chao 1 王晓晨 WANG Xiaochen 1 2 王中元 WANG Zhongyuan 1 胡瑞敏 HU Ruimin 1 武汉大学计算机学院国家多媒体软件工程技术研究中心 武汉大学计算机学院国家多媒体软件工程技术研究中心 National Engineering Research Center for Multimedia Software,School of Computer,Wuhan University National Engineering Research Center for Multimedia Software,School of Computer,Wuhan University 武汉大学深圳研究院 武汉大学深圳研究院 Research Institute of Whuan University in Shenzhen Research Institute of Whuan University in Shenzhen 现有基于视觉信息的说话检测方法中往往依赖预定参数或者阈值作为分类平面,鲁棒性差且泛化能力不强.针对这一问题,本文提出了一种基于显著度的视觉说话检测方法,利用像素的色彩信息和嘴唇的空间位置特性检测嘴唇,通过分析嘴唇运动和说话假设的关系,将图像能量作为特征,并结合经典的隐马尔科夫模型（hidden Markov models,HMM）和支持向量机（support vector machine,SVM）作为判决方法来进行检测.实验结果表明,本文提出的嘴唇检测方法正确率可达到92%. Lip activity detection can be thought of as the visual counterpart of voice activity detection.However,previous visual based voice activity detection（V-VAD）methods usually consist of classifications using a heuristically defined thresholding which makes the V-VAD approaches vulnerable to noisy observation and environmental changes,and leads to poor performance and robustness in the condition of undesired frequent speaking state changes.In this paper,we introduce a novel mixed approach based on combining the relationship between lip movements and speech assumes with hidden Markov models（HMM）and support vector machine（SVM）,which calculates the image energy of lip regions detected by saliency of pixels’ color information and spatial characteristics.Compared with traditional VVAD methods,the correct detection rate of our algorithm also reaches 92%. 显著度检测 视觉说话检测 图像能量 隐马尔科夫模型 支持向量机 saliency detection visual voice activity detection image energy hidden Markov models support vector machine 10.14188/j.1671-8836.2015.04.011 TP391.41 国家自然科学基金重点项目（61231015,61303114）;国家高技术研究发展（863）计划（2013AA014602）资助项目;教育部高校博士点基金新教师类项目（20130141120024）;湖北省自然科学基金面上项目（2014CFB712） 2015-04-01 2021-03-31 4 61