Computer-aided analysis is helpful in improving heart sound classification. PhysioNet Challenge 2022 provides a platform for researchers to evaluate their proposed classification algorithms. In the Challenge, our team (HearTech+) proposed a recording quality assessment method based on frequency density distribution for label correction to prevent the poor-quality recording segments from misleading network optimisation. Besides, a hierarchical multi-scale convolutional neural network (HMS-Net) was designed to conduct both the murmur $(T1)$ and clinical outcome $(T2)$ classification. HMS-Net extracts convolutional features from the spectrograms on multiple scales and fuses them through its hierarchical architecture. The network builds long short-term independencies between multi-scale features and improves the classification performance. Finally, the prediction of a patient is based on the ensembled segment predictions by sliding window. In the five-fold cross-validation by patients, the proposed algorithm performed an average weighted accuracy of 0.81 (best 0.853) on $T1$ and an average challenge score of 9808 (best 9242) on $T2$. In the Challenge hidden validation set, the proposed algorithm achieved 0.806 weighted accuracy on $T1$ and 9120 challenge score on $T2$, ranking 1 st and $4^{th}$ out of 305 entries, respectively. In the final hidden testing set, $T1$ was 0.776 ranking $2^{nd}$, and $T2$ was 12069.