The integration of distributed energy systems into grids increases the uncertainty of electric loads. Accurate short-term load forecasting is critical to cope with the uncertainty and secure the operation of power systems. In this article, we propose a short-term probabilistic load forecasting model based on the quantile regression neural network (QRNN) and an accumulated hidden layer connection (AHLC) structure. The AHLC structure connects the hidden layers of all the predicted hours and can provide more information to the model output layers. This AHLC structure, together with parallel prediction structure and 1-D convolutional structure, improves the accuracy of the short-term probabilistic load prediction. Adaptive fuzzy control is employed to rectify data anomalies caused by emergency situations. The proposed model has been evaluated using the publicly available GEFCom2014 dataset, the ISO-NE dataset, and the Malaysia dataset. Numerical results show that the proposed AHLC-QRNN model has better performance compared to existing models.