Unmanned aerial vehicles (UAVs) are widely employed in wireless communication networks due to their remarkable flexibility and mobility. In this work, we study a multi-UAV-assisted data collection system, where multiple UAVs cooperatively harvest data from ground users (GUs). Aiming at minimizing the mission completion time, we first formulate the trajectory design problem as a min-max optimization problem, subject to the constraints of energy consumption and collision avoidance. Then, by applying the popular QMIX framework, we propose a multi-agent deep reinforcement learning (DRL) algorithm to optimize the trajectory of all UAVs. With the well-designed reward function, the proposed multi-agent DRL approach can coordinate all agents to achieve the final objective based on their local observations. Simulation results verify the superiority of the proposed multi-agent DRL approach over the benchmark algorithms in terms of the mission completion time.