In recent years, deep learning has been widely used in the field of medical image processing, such as identification of symptoms, detection of organ. Due to the complexity of medical images, in the model training, there are many parameters when deep learning is used for image classification, and it takes a long time. Therefore, the training process for neural networks needs to be optimized. The stochastic gradient descent with momentum (SGD) is a common optimization algorithm in deep learning, and the particle swarm optimization (PSO) is a classical and effective swarm intelligence optimization algorithm. These two methods have their own advantages and disadvantages. Combining the two algorithms to calculate parameters, this paper proposes a novel particle swarm optimization-stochastic gradient descent with momentum (PSO-SGD) algorithm, which can find the optimal solution of the network more quickly and improve the solution efficiency on the basis of ensuring the classification accuracy. This algorithm is verified on two data sets, namely Blood Cell Images Data Set (BCIDS) and COVED-19 Radiography Data Set (COVED19RDS). Experiments prove the effectiveness of the algorithm.