The process of automatically tracking people within video sequences is currently receiving a great deal of interest within the computer vision research community. In this paper we contrast the performance of the popular Mean-Shift algorithm’s gradient descent based search strategy with a more advanced swarm intelligence technique. Towards this end, we propose the use of a Particle Swarm Optimization (PSO) algorithm to replace the gradient descent search, and also combine the swarm based search strategy with a Probabilistic Data Association Filter (PDAF) state estimator to perform the track association and maintenance stages. Performance is shown against a variety of data sets, ranging from easy to complex. The PSO-PDAF approach is seen to outperform both the Mean-Shift + Kalman filter and the single-measurement PSO + Kalman filter approach. However, PSO’s robustness to low contrast and occlusion comes at the cost of higher computational requirements.