Improved POMDP Tree Search Planning with Prioritized Action Branching
- Resource Type
- Authors
- Mern, John; Yildiz, Anil; Bush, Larry; Mukerji, Tapan; Kochenderfer, Mykel J.
- Source
- Proceedings of the AAAI Conference on Artificial Intelligence. 35:11888-11894
- Subject
- FOS: Computer and information sciences
Computer Science - Machine Learning
Artificial Intelligence (cs.AI)
Computer Science - Artificial Intelligence
I.2.8
General Medicine
Machine Learning (cs.LG)
- Language
- ISSN
- 2374-3468
2159-5399
Online solvers for partially observable Markov decision processes have difficulty scaling to problems with large action spaces. This paper proposes a method called PA-POMCPOW to sample a subset of the action space that provides varying mixtures of exploitation and exploration for inclusion in a search tree. The proposed method first evaluates the action space according to a score function that is a linear combination of expected reward and expected information gain. The actions with the highest score are then added to the search tree during tree expansion. Experiments show that PA-POMCPOW is able to outperform existing state-of-the-art solvers on problems with large discrete action spaces.
Comment: 7 pages