The impulse-radio ultra-wideband (IR-UWB) radar has excellent range resolution and penetration, and excels in vital sign monitoring (i.e., respiratory and heart rate). At present, the majority of approaches merely split the echoes into several components and then manually localize the vital signs signal. In this paper, the maximum a posteriori (MAP) estimation-based joint expectation maximization (EM) and particle swarm optimization (PSO) (MEBJEP) method is proposed. Based on the MAP estimation, we combine the EM algorithm and PSO to estimate the frequency and amplitude of vital signs. The experimental results demonstrated that the proposed method is capable of capturing more reliable data in a non-contact way from a probabilistic perspective, compared to the commonly used chirp Z-transform and moving target indicator (CZT-MTI) method and the variational mode decomposition (VMD) method. This work illustrates the value of non-contact vital signs monitoring via IR-UWB radars in the body remote sensing field.