In the previous non-parametric denoising algorithms, there are problems in that the denoised curves deviate from the distribution of the expected result and are not smooth enough. In this paper, we propose a regularized non-parametric denoising method (RegNP) to tackle it. Aiming to suppress the shift of the curve, we introduce a regularization term to the objective function. For the improvement of smoothness, move the improved initialized data by gradient descent. During the experiments, the relationship between the standard deviation of the estimated noise and the optimal coefficient of the regularization term is established. We also propose a new workflow for denoising the signals with spikes. Experimental results indicate that compared with state-of-the-art methods, RegNP has achieved 3% to 53.6% performance gain under the root-mean-square-error(RMSE) metric.