Haze can reduce the visibility of the captured image, making it hard to accurately distinguish the details of each object in the captured image scene. Aiming at the problem of detail loss in existing dehazing models, this paper proposes a lightweight end-to-end image dehazing framework called DFE-GAN (Detail Feature Enhancement-GAN). The missing detail contours in the haze image can be predicted by employing a densely connected detail feature prediction network. Supplemented with a patch discriminator and an improved loss function, the restoration of details in the dehazing image is enhanced to improve image quality. We apply inverse residual modules to extract and fuse multi-scale features from images, which can ensure the real-time processing capability of the model. Compared with previous state-of-the-art approaches, solid experimental results on various benchmark datasets validate the robustness and effectiveness of our model.