Driving fatigue can lead to the decline of driving skills, which cause traffic accidents and endanger the safety of lives and property of people. Although various methods for driving fatigue detection have been put forward, the balance between practicality and accuracy is a major problem in driving fatigue detection at present. Considering the particularity of EEG acquisition equipment, we proposed a strategy for utilizing three prefrontal EEG channels for the detection of driving fatigue based on deep learning (ResNet3D models). Wavelet scale maps of the processed EEG were as the input to the models, and the Bayesian optimization algorithm was utilized to optimize the hyper-parameters of models. The result indicated the ResNet3D-18 model with the recognition accuracy of 79.45 percent performed better than the recognition accuracy of 77.13 percent of the ResNet3D-34 model. Our findings showed that combining three prefrontal EEG channels with ResNet3D-18 model would be an efficacious and practical method to detecting driving fatigue.