The objective of this paper is to develop an autonomous multi-agent system, called artist, which is based on behavior control architecture and is capable of doing reinforcement learning adaptation to environmental changes. Artist uses ART-based AHC, a reinforcement learning architecture, as its inner architecture of a behavior and a coordinator. Based on this architecture, it has advantages of systematic design, learning capability, adaption, homogeneous architecture, etc. We have developed three primitive motion control agents (behaviors), and two coordinator agents (coordinators). They are also implemented both in simulations and in physical experiments.