Multi-task Self-Supervised Adaptation for Reinforcement Learning
- Resource Type
- Conference
- Authors
- Wu, Keyu; Chen, Zhenghua; Wu, Min; Xiang, Shili; Jin, Ruibing; Zhang, Le; Li, Xiaoli
- Source
- 2022 IEEE 17th Conference on Industrial Electronics and Applications (ICIEA) Industrial Electronics and Applications (ICIEA), 2022 IEEE 17th Conference on. :15-20 Dec, 2022
- Subject
- Communication, Networking and Broadcast Technologies
Components, Circuits, Devices and Systems
Power, Energy and Industry Applications
Robotics and Control Systems
Signal Processing and Analysis
Industrial electronics
Heuristic algorithms
Reinforcement learning
Self-supervised learning
Benchmark testing
Feature extraction
Multitasking
reinforcement learning
policy adaptation
policy generalization
self-supervised learning
- Language
- ISSN
- 2158-2297
Policy adaptation remains one of the key challenges for reinforcement learning (RL). Thus, RL agents often fail to generalize to unseen scenarios. In this paper, we propose to improve the generalization of RL algorithms through multi-task self-supervised adaptation (MSSA). The proposed method is a general paradigm that can be implemented on top of any RL algorithm. It better extracts high-level feature representations from augmented observations through incorporating multiple self-supervised learning tasks with complementary objectives. The selected self-supervision tasks include rotation prediction, inverse dynamics prediction and contrastive learning. It then performs control actions based on the extracted features. The proposed MSSA method consistently outperforms all the baseline methods on diverse complex tasks in the DeepMind Control suite benchmark and sets new state-of-the-art results without incurring longer inference time. It is demonstrated that MSSA has superior generalization capability and is robust to environmental changes.