3D object detection plays a vital role in the perception system of self-driving cars for it provides accurate structural information and classification of objects in the scene. Recent works leverage pseudo point clouds to compensate for the sparsity of raw point clouds. However, the coarse pseudo point cloud generation brings huge computational costs to the inference process, resulting in inferior inference speed. In this work, we aim to solve two critical yet not well-addressed issues in pseudo point cloud generation, including the loss of 3D detection inference speed and the additional memory cost due to the massive generation of the pseudo point cloud. We propose attention-guided pseudo point cloud generation to direct the focus of pseudo point cloud generation to hazardous regions. In our experiments, our method improves the inference speed by 21.72% and reduces the memory usage of generated data by 90.03%.