This paper proposes a model to detect and solve coverage holes in 5G Radio Access Network (RAN) deployments operating with millimeter waves. The proposed model utilizes a DBSCAN-based detector to identify coverage constrained areas and then proposes the use of Relay User Equipment (RUE) capabilities to extend the RAN coverage. To optimize the activation and deactivation of RUEs, a Deep-Q-Network-based algorithm is proposed, aiming to improve spectral efficiency and decrease the outage probability experienced by network users. The obtained results demonstrate the effectiveness of the model in accurately detecting coverage constrained areas and efficiently solving these issues by means of an effective RUE activation, leading to significant improvements in network performance while minimizing the time that RUEs remain in active mode, which implies potential benefits for MNOs and UE holders and significant energy savings.