Small hepatocellular carcinoma (SHCC) is among the most fatal cancers, and spotting SHCC symptoms in the early stage is vital for conducting timely treatments. Thus, auxiliary detection algorithms have been developed, especially after convolutional neural networks (CNN) made great progress in processing medical images. However, their performance is confined by dataset, resulting in limitations to accurately detect SHCC appearing small and diffusive in CT images. In our work, Self-Attention mechanism has been introduced as the front end and EfficientNet as our backbone network, contributing to a novel SHCC detection algorithm able to automatically spot subtle lesions through our image-wise annotated dataset in a weakly supervised manner. In our model, the Self-Attention module extracts ROI and background features from original CT images and generates weighed feature map to the EfficientNet. In our backbone network, the EfficientNet learns from input feature maps and is weakly supervised under image-wise annotations. With the pre-process of Self-Attention, our data size for EfficientNet has been reduced, thus enhancing learning efficiency and reducing time consumption. After training on over 1.5k CT images, our model has achieved decent detection performance comparing to other state-of-the-art methods while remaining acceptable complexity.