Brainstem and cerebellar hemangioblastoma (HB) is a rare tumor that presents risks of haemorrhage during biopsy due to the vascular nature of the lesions. However, it remains challenging lor radiologists to distinguish HB from other types of intracranial tumors based solely on MRI scans, due to highly similar imaging characteristics. To address this, we propose a novel patient-level classification frame-work leveraging lesion-aware supervised contrastive learning, named LaSCL-PLC. The core concept is using lesion-focused supervised contrastive learning to extract representations that contain rich information directly related to the tumor region itself. We evaluated LaSCL-PLC on a local dataset of 240 brainstem and cerebellar tumor T1-enhanced MRI scans, including 97 scans positively identified as HB patients. Experiments demonstrate that LaSCL-PLC out-performs current state-of-the-art methods in distinguishing HB, achieving a competitive performance with expert neuroradiologists. The code of our model is now available at https://github.com/Haifengtao/LaSCL-PLC