The existing research on cooperative learning is incapable of ensuring the physical constraints of the systems, resulting in possible damage to the plants. The purpose of this study is to present a cooperative learning control method for multi-agent systems with time-varying output constraints. The backstepping technique is used to design the controller by introducing time-varying barrier Lyapunov functions (BLFs) and radial basis function (RBF) neural networks (NNs). Contrary to previous results, the use of BLFs ensures that time-varying output constraints are never violated during the learning control process. The controller for the same plant is designed using the RBF NN approximation of unknown dynamic functions of the plants along the union orbit of all agents. By utilizing a previously learned RBF NN, the control performance is improved and the computational load is reduced. Finally, a numerical example is given to show the tracking performance with time-varying output constraints, the learning ability of RBF NNs, and the improved performance of the control system with learned RBF Nns.