ENTL: Embodied Navigation Trajectory Learner
- Resource Type
- Conference
- Authors
- Kotar, Klemen; Walsman, Aaron; Mottaghi, Roozbeh
- Source
- 2023 IEEE/CVF International Conference on Computer Vision (ICCV) ICCV Computer Vision (ICCV), 2023 IEEE/CVF International Conference on. :10829-10838 Oct, 2023
- Subject
- Computing and Processing
Signal Processing and Analysis
Location awareness
Representation learning
Computer vision
Navigation
Computer architecture
Predictive models
Data models
- Language
- ISSN
- 2380-7504
We propose Embodied Navigation Trajectory Learner (ENTL), a method for extracting long sequence representations for embodied navigation. Our approach unifies world modeling, localization and imitation learning into a single sequence prediction task. We train our model using vector-quantized predictions of future states conditioned on current states and actions. ENTL’s generic architecture enables the sharing of the the spatio-temporal sequence encoder for multiple challenging embodied tasks. We achieve competitive performance on navigation tasks using significantly less data than strong baselines while performing auxiliary tasks such as localization and future frame prediction (a proxy for world modeling). A key property of our approach is that the model is pre-trained without any explicit reward signal, which makes the resulting model generalizable to multiple tasks and environments. We release the code at https://github.com/klemenkotar/ENTL