An improved image tracking algorithm, PCA tracking algorithm combined with adaptive Kalman filter, is proposed in the paper. The algorithm is used to track targets in image sequences. The PCA tracking algorithm is used to initialize the Kalman filter, and the prediction value is used as the search center of the PCA tracking algorithm. Loop this process to track targets in the image sequence. The targats can be followed very well, for the adaptive Kalman filter is used to predict the trajectory of the object . The PCA tracking algorithm is based on the edge structure, hence when the image gray of the target and background is mixed into one, the targets can be marked. The simulation results show that the algorithm has good capacity to follow the target and has good anti-noise and anti-degradation ability.