A rapid and effective method for detecting fungal infection in choy sum seeds is necessary to ensure good yield. In this study, 127 spectra of healthy choy sum seeds and 1479 spectra of seeds of choy sum infected with Penicillium decumbens (P. decumbens) were collected using a laboratory-built hyperspectral imaging system. The imbalanced distribution of samples was improved using the synthetic minority over-sampling technique (SMOTE) algorithm. Nine classifiers were used as base classifiers; discriminant analysis was selected as the meta-learner to build the stacking ensemble learning model. The synergy interval partial least square (siPLS) algorithm was used to filter characteristic wavelengths. The SMOTE-siPLS-stacking model was developed using two wavelength ranges (460.96–516.33 nm and 696.61–753.55 nm) as input, achieving accuracy, and F1-score of 99.79% and 99.89%, respectively. The results showed that hyperspectral imaging combined with the SMOTE-siPLS-stacking model is a feasible method to detect P. decumbens.Graphical Abstract: