Naming TV characters by watching and analyzing dialogs
- Resource Type
- Conference
- Authors
- Haurilet, Monica-Laura; Tapaswi, Makarand; Al-Halah, Ziad; Stiefelhagen, Rainer
- Source
- 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on. :1-9 Mar, 2016
- Subject
- Computing and Processing
Face
TV
Videos
Support vector machines
Labeling
Data models
Training data
- Language
Person identification in TV series has been a popular research topic over the last decade. In this area, most approaches either use manually annotated data or extract character supervision from a combination of subtitles and transcripts. However, both approaches have key drawbacks that hinder application of these methods at a large scale — manual annotation is expensive and transcripts are often hard to obtain. We investigate the topic of automatically labeling all character appearances in TV series using information obtained solely from subtitles. This task is extremely difficult as the dialogs between characters provide very sparse and weakly supervised data. We address these challenges by exploiting recent advances in face descriptors and Multiple Instance Learning methods. We propose methods to create MIL bags and evaluate and discuss several MIL techniques. The best combination achieves an average precision over 80% on three diverse TV series. We demonstrate that only using subtitles provides good results on identifying characters in TV series and wish to encourage the community towards this problem.