The problem of stability analysis of neural networks (NNs) with time-varying delay is investigated in this paper. By establishing fractional order auxiliary polynomials and introducing slack variables reasonably, some improved fractional order auxiliary-polynomial-based functions (FOAFs) are developed to exploit additional degrees of freedom and more information on extra states. Based on FOAFs, a new Lyapunov-Krasovskii functional (LKF) is constructed, and a new stability criterion for NNs with time-varying delay is proposed. In addition, by adding augmented terms on the proposed LKF, we obtain an improved stability criterion for NNs with time-varying delay. Finally, a numerical example is given to illustrate the advantages and effectiveness of the proposed methods.