In this paper, we investigate a multi-UAV communication system with moving users and consider the co-channel interference caused by the transmissions of all other UAVs. To ensure the fairness of moving users, we maximize the minimum average user rate during the observed time by jointly optimizing UAVs' trajectories, transmission power, and user association. To effectively tackle this non-convex problem with both discrete and continuous variables, we propose a joint neural network (NN) design, where a network named advantage pointer-critic (APC) is applied to optimize discrete variables and a deep-unfolding NN is used to optimize continuous variables. Specifically, we first elaborately formulate a Markov decision process to model the user association, and then use the APC network trained by the advantage actor-critic algorithm to address it. As for the deep-unfolding NN, we first develop a block coordinate descent based algorithm to optimize UAVs' trajectories and transmission power, and then unfold this algorithm into a layer-wise NN with introduced trainable parameters. These two networks are jointly trained in an unsupervised fashion. Simulation results validate that the proposed joint NN significantly outperforms the mathematical optimization algorithm with much lower complexity.