There has been a growing attention to the use of photoacoustic sensing techniques for medical applications in recent decades. With respect to the layered photoacoustic imaging (PAI) scenarios (e.g., brain imaging and breast imaging), the plane wave model-based phase shift migration (PSM) method was proposed. However, this classical plane wave model is only suitable for planar layered media and does not consider the influence of the difference of the transverse sound velocity change, which seriously limits its applications in practice. In this work, we propose to use the Fourier finite difference migration (FFD) to perform photoacoustic reconstruction in irregularly layered heterogeneous media. This method employs the phase shift method in the frequency wavenumber domain for vertical field extrapolation and Fourier difference-based “thin lens” correction for lateral speed of sound (SOS) variation compensation. Both simulation and experimental results demonstrate that the proposed method can reconstruct images with highly improved imaging resolution and signal-to-noise ratio. The development of this technology is anticipated to facilitate the acquisition of real-time, high-quality images of biological tissues, such as blood vessels and melanoma.