Infrared temperature measurement is a widely used non-contact measurement method in power system, which can accurately measure the temperature of power equipment, find the temperature anomaly of equipment in time, and then judge the operation status of equipment. The local heating of power equipment mainly includes the heating caused by the unfastened connection of the transformer bushing, the heating caused by the poor crimmage of the voltage connector of the transmission line, and the heating caused by the unfastened external connection of the current transformer. For the infrared image taken, there are often problems such as complex background, inaccurate target recognition, and fuzzy temperature recognition. Therefore, we optimize the yolov5 multi-target detection and temperature measurement method. Multi-modal data fusion is constructed by RGB image and infrared image to improve the robustness of the model and strengthen the target fine detection. In the temperature measurement scheme, the K-means algorithm is used to cluster the infrared image of the electrical equipment, and the temperature information contained in each tiny area in the target is displayed, which is convenient to find the temperature hidden danger. The comparison results show that the method has achieved a better temperature diagnosis effect.