Abstract Background We designed a new surgical procedure to treat benign prostatic hyperplasia(BPH). In order to verify its effectiveness and safety, we constructed this randomized controlled trial to compare the efficacy of our innovative enucleation technique- photoselective sharp enucleation of the prostate (PSEP), with a front-firing 532-nm laser and the traditional technique-photoselective vaporization of the prostate (PVP) in the treatment of BPH. Methods A total of 154 consecutive patients diagnosed with bladder outlet obstruction secondary to BPH in our center from June 2018 to April 2019 were randomly divided into the PSEP group (n = 77) and the PVP group (n = 77) and were treated surgically with either PSEP or PVP. All patients were assessed preoperatively and followed up at 1, 6, and 12 months postoperatively. The international prostate symptom score,quality-of-life score, postvoid residual urine volume, maximum urine flow rate, prostate volume, prostate-specific antigen, and adverse events were compared. Results The lower urinary tract symptoms in both groups were significantly improved compared with the baseline at 1, 6, and 12 months postoperatively. The PSEP and PVP groups had an equivalent International Prostate Symptom Score, quality-of-life score, postvoid residual urine volume, maximum urine flow rate, prostate-specific antigen at each follow-up (P > 0.05). The median operative time in the PSEP group was significantly shorter than that in the PVP group (35 min vs. 47 min, P