To address the problems of low precision and inaccurate detection of cartoon character face recognition based on traditional network model, our paper proposes an improved YOLOv8 detection model based on YOLOv8n optimization. Firstly, C2f-DCN module is used to replace part of C2f, which improves the detection performance of the model; Secondly, the GAM attention module is added in the main part to strengthen the semantic information and position information in the features, which improves the feature fusion ability of the model; Finally, WIoU boundary loss function is used to replace the original loss function, which improves the bounding box regression performance of the network. The experimental results show that the detection precision of the improved model in the dataset Manga109-Face is 91%, the recall rate is 88.3%, the mAP is 93%, and the number of parameters is 4. 8*10 6 ; Compared with YOLOv8, the detection precision, recall and mAP are increased by 1.5%, 2.3% and 2.1%, respectively. Result shows that the proposed model improves the detection precision and has practical application value.