• Combination of transmittance and interactance modes was presented. • Variable selection methods affected the performance of PLS-DA model. • Combination of NIR modes improved the performance of PLS-DA model. • The different types of misclassified samples were analyzed. In this research, a novel non-destructive detection mode of moldy apple core was proposed. A spectral acquisition system based on mini-spectrometer technology for Fuji apple was built, it combined interactance and transmittance modes of Vis/NIR spectroscopy. Spectral data were analyzed by variable selection methods—competitive adaptive reweighted sampling (CARS) and successive projection algorithm (SPA)—to extract feature wavelengths related to moldy apple core. The CARS-PLS-DA model that was established on the basis of spectral data obtained by combination of interactance and transmittance modes has the best performance. The recall, specificity and accuracy in the prediction set were 87.88%, 90.91% and 89.39%, respectively. Compared with the interactance modes, recall, specificity and accuracy of the prediction set are increased by 9.09%. Compared with the transmittance modes, recall and accuracy of the prediction set are increased by 3.03% and 1.51%. The mini-spectrometer technology based spectral acquisition system that combined interactance and transmittance modes of Vis/NIR spectroscopy effectively realized complementary advantages of each mode and improved the performance of PLS-DA model. [ABSTRACT FROM AUTHOR]