To address the problem that insulator small faults in transmission lines are difficult to identify and combine the features of fault images, this paper proposes a fault detection algorithm based on improved YOLOv8 for insulator small targets. Firstly, we use the C2f-DCN module in the backbone extraction network to solve the problem that it is difficult to obtain the features of small targets due to the image deformation caused by compressed pixels. Secondly, we add the BiFormer attention module at the bottom of the backbone network, which can focus on the small fault features of insulators in the complex background and improve the feature representation capability of the network. According to the experiments, the detection accuracy of the improved model in this paper is 93.2%, the recall rate is 84.3%, the mAP is 91.8%, the number of parameters is 34.2×10 6 , and the number of floating point operations (FLOPs) is 17.9G; compared with YOLOv8, the detection accuracy, recall rate, and mAP are improved by 4.1%, 3.2%, and 3.8%, respectively, and the results show that the algorithm is the complex environment has significantly improved the detection accuracy of small targets.