A study on agricultural automation became more important. In Korea, sweet persimmon farmers spend a lot of time and effort on classifying profitable persimmons. In this paper, we propose and implement an efficient grading algorithm for persimmons before shipment. We gathered more than 1,750 images of persimmons, and the images were graded and labeled for classifications purpose. Our main algorithm is based on EfficientDet object detection model but we implemented more exquisite method for better classification performance. In order to improve the precision of classification, we adopted a machine learning algorithm, which was proposed by PyCaret machine learning workflow generation library. Finally we acquired an improved classification model with the accuracy score of 81%.