The pipeline system is available to process multiple training batches simultaneously. Thereby, it can allow forward and backward propagation tasks for each batch to be performed across multi-time units on multi-compute nodes. The pipeline system can ensure better utilization of computing resources. However, a large amount of memory consumption during training is attributable to the large number of feature maps, leaving the performance of the pipeline system not fully utilized. To address the problem of high memory usage in pipeline parallel training, we propose a pipelined memory optimization approach based on feature maps encoding (FePipe). FePipe encodes the feature maps in the computing intervals between forward and backward propagation. Then, to reduce the memory usage, it uses binarization encoding to store the redundant elements of the input feature maps in the combined Relu-Pooling layer and CSR encoding to store the sparse feature maps in the combined Relu-Conv layer. It can be seen from the experimental results that, compared with the data parallel approach BSP and the pipeline parallel training approach PipeDream, the average memory usage of FePipe decreases by 1.51 times and 1.33 times, respectively. Besides, FePipe can effectively complete the training task with a small loss of accuracy.