Infrared moving target tracking algorithm in complex background prone target loss problems, proposed to add background perception module and target feature matching function in tracking algorithm, real-time analysis of the surroundings of the target. Strict constraints target infrared feature matching function when found that the background will be changed, if the target is interference, and outputs the target predicted position until the target flight out of the area. The improved algorithm is substantially independent of the target size, rotation change and improves the stability and reliability of the tracking algorithm. The experimental results show that the improved algorithm to overcome easily lost in the traditional algorithm to track the target and stable tracking in a complex background.