To address the problem of complex texture and smallness of defect in composite surface defect, this paper improved the detection model of features based on the U-Net architecture and a detailed analysis of the attention model is made. First, features contraction completed through the Encoder (the left path), then features expansion and recovery completed through the Decoder (the right path). The improved model introduces the channel attention model and the spatial attention model. The residual model is introduced to reduce the impact of network degradation and gradient disappearance on the model. Depth-wise separable convolution is introduced to lightweight the model. The results of comparative testing in ablation experiments show that the improved detection model can accurately identify defects, especially the smallness defect. It is much better if you place the channel attention in front of the spatial attention and selecting $7\times 7$ as the kernel size of the convolution in spatial attention. The total pixel accuracy of the model detection in this paper reaches 95.42% and the intersection over the union (IoU) value of the proposed algorithm reaches 84.98% which can satisfy the effective identification of composite surface defect detection and has strong robustness.