Learning to Follow Verbal Instructions with Visual Grounding
- Resource Type
- Conference
- Authors
- Unal, Emre; Can, Ozan Arkan; Yemez, Yucel
- Source
- 2019 27th Signal Processing and Communications Applications Conference (SIU) Signal Processing and Communications Applications Conference (SIU), 2019 27th. :1-4 Apr, 2019
- Subject
- Communication, Networking and Broadcast Technologies
Signal Processing and Analysis
Flyback transformers
Voltage control
autonomous agents
navigational instruction following
natural language processing
computer vision
- Language
We present a visually grounded deep learning model towards a virtual robot that can follow navigational instructions. Our model is capable of processing raw visual input and natural text instructions. The aim is to develop a model that can learn to follow novel instructions from instruction-perception examples. The proposed model is trained on data collected in a synthetic environment and its architecture allows it to work also with real visual data. We show that our results are on par with the previously proposed methods.