Increasing demands for artificial intelligence tools in the medical sector show the huge interest of physicians. In recent years, one of the most effective assistant systems in this field is the real-time detection of early-stage colorectal polyps during colonoscopy. Since even experienced physicians may miss polyps during colonoscopy, the real-time assistance system is designed to prevent this and hence contributes to diminish the number of missed critical cases. One challenge in this field is the detection of false positives. These systems are prone to mistake artifacts for colorectal polyps. This review provides an overview over current quality assessment and restoration techniques to make a high-quality training dataset for training deep neural network algorithms. Furthermore, four of the latest, fastest, and most accurate methods are introduced and analyzed in the rest of the review. Our main contribution is to provide an analysis of current methods used to detect colorectal polyps. We present a list of available datasets and present a range of challenges colorectal cancer detection systems face.