Single-cell Hi-C technology is utilized to obtain chromosome interaction information at the single-cell level and further study the differences in genome structures between different cell types. However, there are few accurate and efficient clustering methods for single-cell Hi-C data, with the following manifestations: The clustering efficacy on the dataset with a large number of cells is not very satisfactory, and it is difficult to cluster these cells with small number in the whole dataset. In this study, we propose a high-performance single-cell Hi-C clustering framework, called scHiCSC. A new smoothing method based on contact number weight is first proposed to generate cell embedding with more accurate cell features. In addition, a novel feature fusion method is proposed to further supplement the feature information of cells by fusing the chromosome structure information within cells and the distance information between cells. The experimental results show that scHiCSC has a strong generalization ability on different sizes of datasets and outperforms the existing single-cell Hi-C clustering frameworks. Moreover, scHiCSC achieves an optimal and stable clustering efficacy in the datasets with large-scale cell numbers and can cluster the cells with small number in the whole dataset. The source code of scHiCSC is freely available at https://github.com/HaoWuLab-Bioinformatics/scHiCSC.