In order to reduce the data storage and improve data compression ratio of the stiffness matrix of 3D finite element, after analyzed the relationship between nonzero submatrix and generalized adjacent nodes of the stiffness matrix, this paper proposes an improved stiffness matrix compression algorithm, which combined negative sign compressed sparse line and a rider to store binary classification method. Then the improved algorithm is applied to the storage of the stiffness matrix of 3D-FEM. Through experimental simulation, the results show that this method saves a lot of storage space to ensure the validity of data for finite element analysis.