Various types of road damage occur frequently, which can affect the smooth running of vehicles. The detection of road surface damage is of great significance for road surface maintenance and smooth traffic flow. First, this paper makes descriptive statistics on RDD2020 dataset, and deals with the mislabeled categories in the dataset, through which 14,569 samples are obtained. A single-stage object detection network YOLOv5 is then constructed to detect road damage on the RDD2020 dataset. The experiment results show that the proposed network is effective in road damage detection of RDD2020 dataset. Faced with high-cost detection methods, a convenient and efficient road damage detection network is urgently needed. In this paper, a road damage detection system is deployed, which can detect the location of road damage and identify the types of road damage in real-time under the camera shooting.