Implicit correlations between lesions in medical images are the key to improve the performance of automatic medical lesion annotation tasks, as additional information they provide can improve the accuracy of the annotation results. Existing methods use single-scale features to model the correlations. However, single-scale features are not conducive to the annotation of small and variable scale lesions. In this paper, we propose a feature fusion correlation network with a feature fusion module and a correlation learning module. The feature fusion module refines the feature maps extracted from different blocks of the backbone and obtains multi-scale features, while the correlation learning module uses these multi-scale features to capture the correlations between different scales of lesions. We conduct experiments on an in-house dataset, and the experiment results show that our method achieves the best performance.