Traffic sign detection and recognition plays an important role in driver assistance system especially in complexity environment. Firstly, RGB image is converted to standardization image only contained 8 colors for reducing computational burden. Only interesting color components are extracted as candidate region for further recognition. Then HOG descriptor is considered as detection characteristic of candidate regions. Finally, Zernike moment is combined with support vector machine for traffic sign recognition. Experiments show that it could defense of shadow, rainy days, and dim by take advantage of color standardization and Zernike moments.