Deep learning (DL) is widely used in the field of fault diagnosis. The training of DL-based fault diagnosis methods commonly requires the collection of comprehensive datasets that include all fault classes; however, new fault classes will continue to emerge during diverse phases of the service time of rotating machinery. Traditional DL-based models suffer from reduced diagnostic accuracy due to their inability to adaptively identify new fault classes; therefore, a distribution character-guided projection replay network (DCGPR) is proposed for such fault incremental diagnosis of rotating machinery. First, a distribution projection replay (DPR) module is designed to store the distribution information of former fault classes and replay the general fault knowledge in the next incremental training stage. Second, a prototype adaptive update (PAU) module is further developed to avoid biased prediction of frequent parameter updates faced by existing fault incremental diagnosis methods. Finally, experimental results on the bearing fault dataset and gearbox fault dataset verified the superiority of the proposed method over the latest incremental fault diagnosis methods.