The statistics of the number of personnel on the construction main working face is one of the important indicators for identifying construction progress, and is of great significance for the refined management and resource allocation of construction projects. The deep learning object detection algorithm can effectively replace the existing manual statistics method, reduce mechanical and time-consuming statistical work, and improve statistical accuracy. However, existing object detection algorithms have technical shortcomings such as information loss, insufficient features, and scale imbalance in detecting such small-sized entities. Therefore, this study improves the YOLOv5 object detection algorithm through methods such as image enhancement, multi-scale samples, data enhancement, and model improvement. The experimental results show that our improved algorithm achieves a mean average precision of 83.3%, significantly improving accuracy and robustness, and effectively solving the problem of detecting small and medium-sized personnel in the captured images of the construction main working face.