The cooperative improvement of strength and toughness of rigid polyurethane (RPU) is a vital requirement as a structural material. In this work, 0.2 wt% short carbon fibers (CFs) reinforced RPU composites with high strength and toughness were prepared. The effects of fiber surface treatment and the mechanical properties of RPU composites were studied. Scanning electron microscopy (SEM) was performed to observe the surface morphology of the CFs as well as the fracture surfaces of the composites. Fourier-transform infrared spectroscopy (FT-IR) and contact angle instrument were used to analyze the surface functional groups and the surface energy of CFs, respectively. The results indicated that the surface roughness, the content of OH groups and the surface energy of carbon fibers increased after oxidation modification. Compared with pure RPU, the tensile, bending and impact strength of modified CFs/RPU composites were increased by 38.8%, 22.4% and 71.4%, respectively. Dynamic thermomechanical analysis (DMA) and SEM indicated the positive effects of oxidation modification on the interfacial bonding between modified CFs and RPU matrix.