Although great progress has been made in object detection, especially face detection, in recent years, one of the remaining difficult problems is maintaining excellent performance while detecting in real-time. To address this problem, we propose an effective single-stage object detector named Look Once Rapid Object Detector (LOROD), which uses fully convolutional layers. In particular, our framework is consists of two major modules. One is the Residual Convolutional Layer (RCL) module, which uses ResBlocks to achieve rapid convergence. The other module, namely the Multiple Scale & Dense Connection Convolutional Layers (MS-DCCL) module, is designed to enrich the feature maps and adapt to different scales. In addition, we propose some date augment strategies to increase the model’s robustness. When running on GTX1080Ti, given images with a resolution of 352×352, the average time consumed for inference is merely 15.1 ms and the accuracy can reach 0.946 on the FDDB dataset. By comparing with the advanced algorithms, our method’s effectiveness is proved.