The Effect of Real-Time Constraints on Automatic Speech Animation
- Resource Type
- Authors
- Danny Websdale; Sarah Taylor; Ben Milner
- Source
- INTERSPEECH
- Subject
- 0301 basic medicine
Context effect
Computer science
Speech recognition
Latency (audio)
Window (computing)
020207 software engineering
Context (language use)
02 engineering and technology
Animation
03 medical and health sciences
030104 developmental biology
Recurrent neural network
0202 electrical engineering, electronic engineering, information engineering
Coarticulation
Computer facial animation
- Language
- English
Machine learning has previously been applied successfully to speech-driven facial animation. To account for carry-over and anticipatory coarticulation a common approach is to predict the facial pose using a symmetric window of acoustic speech that includes both past and future context. Using future context limits this approach for animating the faces of characters in real-time and networked applications, such as online gaming. An acceptable latency for conversational speech is 200ms and typically network transmission times will consume a significant part of this. Consequently, we consider asymmetric windows by investigating the extent to which decreasing the future context effects the quality of predicted animation using both deep neural networks (DNNs) and bi-directional LSTM recurrent neural networks (BiLSTMs). Specifically we investigate future contexts from 170ms (fully-symmetric) to 0ms (fullyasymmetric …