An automatic surface defect of Steel Ball detection system based on the CCD image processing technology has been designed. The system consists of automatic feeding stuff mechanism, feeding stuff expansion, defect recognition and sorted mechanism. We designed improved sabel algorithm for edge detection, and the modified medium filtering technology is used to eliminate noises, and the vector invariance method is applied to automatically divide threshold value. The system can run steadily for long time, and its results are reliable, and it overcomes disadvantages of conventional detecting methods such as high labor intensity and high misdetection rate. Measured data can be automatic acquisition, judgment, storage