To solve the problem of selecting drones for passive positioning within unmanned aerial vehicle (UAV) swarm and optimizing corresponding trajectories. This article constructs a method for determining and optimizing the trajectory of UAVs based on an improved particle swarm optimization (PSO) algorithm. Firstly, the time difference of arrival (TDOA) positioning principle was introduced and corresponding algorithm models were organized. Afterwards, the objective function and constraint conditions for selecting drones and optimizing flight paths were designed. The correlation between the optimal solutions of the continuous time optimization problem is used to determine the UAV for positioning. This paper constructs the UAV determination process based on similarity screening. At the same time, combined with the characteristics of the problem to be optimized, the Particle Swarm Optimization (PSO) is improved from three aspects: updating the initial position of particles, improving the iteration formula and setting the adaptive termination condition. This paper further constructs the track optimization process based on improved particle swarm optimization. Through simulation experiments and algorithm comparison, it can be seen that the method constructed in this article can determine the drone used for positioning in real-time and optimize its spatial position. Compared to the selected drones and mainstream passive positioning methods, the method in this article reduces errors by more than 60% and 45%.