Due to the advantages of long inspection time and good inspection effect, transmission line inspection robot has gained more attention and application. This paper proposed a transmission line obstacle detection and recognition algorithm (H-CNN) for the line inspection robot, which consists of two modules: the obstacle detection module (YN-Net) and the obstacle recognition and positioning module (M-CNN). When an obstacle is detected by the YN-Net, the M-CNN is activated to identify and locate the obstacle based on a binocular camera. To solve the problem of insufficient datasets, the M-CNN divides the neural network into a feature extraction module and a feature application module, while the feature extraction module is pretrained with the common dataset and the feature extraction module is finetuned with the dataset collected in this paper. The experimental results show that the proposed obstacle detection and recognition system can meet the real-time and accuracy requirements.