The trade-off between efficiency and generalization has been a challenging problem in crowd counting. In this work, we are devoted to suggesting an application-oriented solution to practical device deployment for crowd counting in resource-constrained scenes. In order to optimize the application computation complexity, this paper proposes a counting architecture, called Mobile Lightweight Refine Net (MLRNet), which is targeted for the high-effectiveness of neural networks in real-time crowd counting tasks on light-scale servers or intelligent devices. In the proposed framework, MobileNetV3 is used as the front-end which enables the model of an efficient feature extraction ability with low resource use. Besides, the back-end introduces the design of Light-weight RefineNet, which maintains the overall counting accuracy while compressing the computing costs. Extensive experiments on four mainstream datasets have demonstrated the availability of our proposed network. Particularly, compared with the baseline method, MLRNet achieves higher accuracy while only using around 1/3 of the FLOPS and parameters on three public crowd counting datasets.