A method based on regional correlation and color histogram match was proposed for multi-target adaptive tracking. Firstly, target detection method was used to get the difference images. Then, the same target could be recognized according to the correlation of target regions of consecutive frames and the matching rate of color histogram. Finally, the active value, lifetime and dormancy value of targets were used to carry out long-time target tracking and deal with the problems of missing detection, false detection and target exit. The experiment shows that the method proposed in this paper has reached to a high accuracy rate of 95%, which has good robustness against the processing of missing detection, false detection and target exit.