With the continuous development of smart grid, the intelligent and automation level of power grid production safety supervision management mode needs to be improved. In order to solve this problem, this paper proposes a method based on YOLOv4 to identify the unsafe behaviors related to construction, including wearing safety helmet, working clothes and insulating gloves. In this paper, based on image recognition technology, the behavior analysis model is established to automatically identify whether the operator's behavior meets the safety requirements. The mAP of the final algorithm is 91.38%. In this paper, the judgment basis and logic of safety supervision alarm logic for multi construction scenarios are constructed, which provides a new idea for the application of deep learning algorithm in power intelligent safety supervision.