Due to nonideal conditions in actual applications of inverse synthetic aperture radar (ISAR), some measurements are lost or the received signal is invalid for some time periods. In addition, the received signal is often affected by measurement noise. Hence, it is usually difficult to obtain well-focused images for such sparse aperture ISAR data. To solve this issue, a novel low-rank and patch-based sparse ISAR imaging method called LRPB is proposed in this article. In LRPB, the low-rank property and structure similarity of ISAR image in 3-D space are explored to ensure high-quality image reconstruction. Simultaneously, the noise is also considered in the constraint to achieve better performance. Furthermore, a Lagrange multiplier-based technique is developed to tackle the optimization problem of ISAR imaging by combining the advantages of alternating direction multiplier method. The experimental results of simulation data and real measured data verify the effectiveness of the proposed method, especially at a low signal-to-noise ratio (SNR) and a small number of pulses.