Aiming at the problems of high labor consumption, poor flexibility and relative lag in the safety protection measures in the dangerous area of high-voltage test, an intelligent early warning system for people break in danger area of high voltage test by mistake based on video surveillance is designed. Firstly, YOLOv4 and DeepSORT are used to track and calculate pedestrian trajectories, predict the travel direction and trend ac-cording to the obtained trajectories, and judge and warn the travel state of people according to the early warning rules. Using different scenarios in the high-voltage test site to conduct simulation tests, the results show that The system can effectively identify the state and trend of the movement of people in the test site and surrounding areas, and has the ability to predict the behavior of stray in; When a hidden danger occurs, the test can be stopped in time to ensure the safety of high-voltage test. The design of this system can provide a reference for the establishment of a safe and reliable, flexible and efficient, intelligent and perfect high-voltage laboratory.