Heuristic Q-learning based on experience replay for three-dimensional path planning of the unmanned aerial vehicle
- Resource Type
- Authors
- Ronglei Xie; Yunpeng Ma; Zhe Wu; Yaoming Zhou; Zhijun Meng
- Source
- Science progress. 103(1)
- Subject
- Mathematical optimization
Multidisciplinary
Computer science
Heuristic
020208 electrical & electronic engineering
Q-learning
02 engineering and technology
Construct (python library)
Track (rail transport)
Action selection
Convergence (routing)
0202 electrical engineering, electronic engineering, information engineering
State space
020201 artificial intelligence & image processing
Motion planning
- Language
- ISSN
- 2047-7163
In order to solve the problem that the existing reinforcement learning algorithm is difficult to converge due to the excessive state space of the three-dimensional path planning of the unmanned aerial vehicle, this article proposes a reinforcement learning algorithm based on the heuristic function and the maximum average reward value of the experience replay mechanism. The knowledge of track performance is introduced to construct heuristic function to guide the unmanned aerial vehicles’ action selection and reduce the useless exploration. Experience replay mechanism based on maximum average reward increases the utilization rate of excellent samples and the convergence speed of the algorithm. The simulation results show that the proposed three-dimensional path planning algorithm has good learning efficiency, and the convergence speed and training performance are significantly improved.