A vision-based method for detecting key components and defects is proposed. Firstly, a visible light camera is used to capture images of key components of transmission lines. The feature pyramid networks structure is used to improve the accuracy corresponding to multiple scales and enhance the effect of small target detection. Deep learning algorithms such as the YOLOv3 algorithm are used to unify candidate box extraction, feature extraction, target classification, and target localization in a single neural network. This method can predict the target location and probability directly from the candidate region in the image by features of the whole image. Defects and faults in key components such as insulators, bird nests, dampers, conductors and towers are automatically located and identified.