Mutation-Bias Learning in Games
- Resource Type
- Working Paper
- Authors
- Bauer, Johann; West, Sheldon; Alonso, Eduardo; Broom, Mark
- Source
- Subject
- Computer Science - Machine Learning
Computer Science - Multiagent Systems
Mathematics - Dynamical Systems
Mathematics - Optimization and Control
Quantitative Biology - Populations and Evolution
37N40 (Primary) 91A26 (Secondary)
- Language
We present two variants of a multi-agent reinforcement learning algorithm based on evolutionary game theoretic considerations. The intentional simplicity of one variant enables us to prove results on its relationship to a system of ordinary differential equations of replicator-mutator dynamics type, allowing us to present proofs on the algorithm's convergence conditions in various settings via its ODE counterpart. The more complicated variant enables comparisons to Q-learning based algorithms. We compare both variants experimentally to WoLF-PHC and frequency-adjusted Q-learning on a range of settings, illustrating cases of increasing dimensionality where our variants preserve convergence in contrast to more complicated algorithms. The availability of analytic results provides a degree of transferability of results as compared to purely empirical case studies, illustrating the general utility of a dynamical systems perspective on multi-agent reinforcement learning when addressing questions of convergence and reliable generalisation.