Synthetic Experiences for Accelerating DQN Performance in Discrete Non-Deterministic Environments
- Resource Type
- article
- Authors
- Wenzel Pilar von Pilchau; Anthony Stein; Jörg Hähner
- Source
- Algorithms, Vol 14, Iss 8, p 226 (2021)
- Subject
- Experience Replay
Deep Q-Network
Deep Reinforcement Learning
sample efficiency
interpolation
Machine Learning
Industrial engineering. Management engineering
T55.4-60.8
Electronic computers. Computer science
QA75.5-76.95
- Language
- English
- ISSN
- 1999-4893
78378540
State-of-the-art Deep Reinforcement Learning Algorithms such as DQN and DDPG use the concept of a replay buffer called Experience Replay. The default usage contains only the experiences that have been gathered over the runtime. We propose a method called Interpolated Experience Replay that uses stored (real) transitions to create synthetic ones to assist the learner. In this first approach to this field, we limit ourselves to discrete and non-deterministic environments and use a simple equally weighted average of the reward in combination with observed follow-up states. We could demonstrate a significantly improved overall mean average in comparison to a DQN network with vanilla Experience Replay on the discrete and non-deterministic FrozenLake8x8-v0 environment.