In this paper, we propose an efficient parts surface defect detection method using SVM algorithm, and particle swarm optimization is used to make parameters selection for SVM. The proposed parts surface defect detection systems is made up of six parts, and the main ideas of our method lie in that we exploit computer vision and machine learning in the research field of mechanical manufacturing and automation. We convert the parts surface defect detection problem to the classification problem, and the images of parts surface are used as testing samples. The SVM algorithm regards the classification problem as the constrained optimization problem. The classification accuracy is determined by the quality of parameters selection. Hence, particle swarm optimization is exploited to make parameters selection for SVM by defining two fitness functions. Afterwards, the best particle of the current population and the gbest is obtained. Utilizing the output from the particle swarm optimization then the parameters for SVM can be obtained. Finally, experiments are conducted based on a dataset with 563 samples, and experimental results illustrate that the proposed is quite effective for parts surface defect detection.