A crop monitoring system is an alternative for securing stable quantities and improving the quality of soybeans. In particular, leaves are closely related to crop yield and quality, as they provide important diagnostic properties for most plant conditions. However, since the existing monitoring method is performed manually, it is inefficient because it consumes time and labor. Monitoring technology using Convolutional Neural Network (CNN) may supplement the limitations of existing monitoring methods as it can help monitor crop conditions efficiently and conveniently, saving time and labor. In addition, this monitoring technology is promising because it can be used in various fields such as plant classification, pest detection, and weed detection. In this study, I aim to (1) propose an optimized CNN model that classifies soybean varieties through leaf image data, and (2) build a system for detecting damage to soybean leaves by pests. Daepung and Pungsannamul leaf and canopy image data were obtained from the soybean fields at Chonnam National University, Gwangju and National Institute Crop Science, Wanju-gun, Jeollabuk-do from 2021 to 2022. All datasets consist of images with complex backgrounds such as soil, weeds, and other plants acquired in various weather conditions such as sunny, cloudy, and rainy days. I obtained 4,827 leaf images for classification and 795 soybean images for detection. The ResNet50 model selected for classifying the varieties of Daepung and Pungsannamul showed strong performance in feature extraction. This model was optimized through hyperparameters adjustment, and classification was performed by attaching a sigmoid activation function. ResNet50 reached an accuracy of 94.5 % and also showed the best performance when compared to the other four models (VGG16, Inception-V3, MobileNetV3, and NASNet). Using the optimized ResNet50 as a feature extractor, I built a Faster R-CNN model to detect leaves damaged by pests at the community level. The Faster R-CNN model consists of a feature extraction area composed of the ResNet50 model and Feature Pyramid Network, a Region Proposal Network area that recommends regions, and a classifier area from which results are derived. This model reached a mean average precision of 72.6 %. In conclusion, I showed that soybean growth diagnosis could be achieved with a high level of accuracy in real fields using the CNN-based models proposed in this study.