Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Visual speech encoding based on facial landmark registration

Krish,, Ram P. and Whelan, Paul F. orcid logoORCID: 0000-0001-9230-7656 (2016) Visual speech encoding based on facial landmark registration. In: Irish Machine Vision & Image Processing Conference 2016, 25-26 Aug 2016, Galway, Ireland. ISBN 978-0-9934207-1-9

Visual Speech Recognition (VSR) related studies largely ignore the use of state of the art approaches in facial landmark localization, and are also deficit of robust visual features and its temporal encoding. In this work, we propose a visual speech temporal encoding by integrating state of the art fast and accurate facial landmark detection based on ensemble of regression trees learned using gradient boosting. The main contribution of this work is in proposing a fast and simple encoding of visual speech features derived from vertically symmetric point pairs (VeSPP) of facial landmarks corresponding to lip regions, and demonstrating their usefulness in temporal sequence comparisons using Dynamic Time Warping. VSR can be either speaker dependent (SD) or speaker independent (SI), and each of them poses different kind of challenges. In this work, we consider the SD scenario, and obtain 82.65% recognition accuracy on OuluVS database. Unlike recent research in VSR which makes use of auxiliary information such as audio, depth and color channels, our approach does not impose such constraints.
Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Uncontrolled Keywords:computer vision; image analysis; Visual Speech Encoding; Facial image analysis; Landmark Registration
Subjects:Computer Science > Machine learning
Computer Science > Image processing
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Electronic Engineering
Published in: Devaney, Nicholas, (ed.) Proceedings of the Irish Machine Vision & Image Processing Conference 2016. . Irish Pattern Recognition & Classification Society (IPRCS). ISBN 978-0-9934207-1-9
Publisher:Irish Pattern Recognition & Classification Society (IPRCS)
Official URL:http://hdl.handle.net/10379/6136
Copyright Information:© 2016 Irish Pattern Recognition & Classification Society (IPRCS)
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:RESPECT & the People Programme (Marie Curie Actions) of theEU’s 7th Framework Programme (FP7/2007-2013) REA grant no: PCOFUND-GA-2013-608728.
ID Code:22091
Deposited On:27 Oct 2017 12:06 by Paul Whelan . Last Modified 11 Jan 2019 10:31

Full text available as:

[thumbnail of RamKrish_IMVIP_2016.pdf]
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader


Downloads per month over past year

Archive Staff Only: edit this record