Due to the factor of energy structure, coal will be the main energy in China for a long time in the future. With mechanized mining, gangue will be mixed into the coal. In order to improve the efficiency of coal production and realize intelligent sensing, the weight statistics of coal and gangue on the conveyor belt have become a hot research topic for many researchers. At present, the weight statistics of coal and gangue are mainly carried out through washing analysis in the subsequent processing links, but the measurement results cannot be timely fed back and guide the optimization of mining work, which restricts the construction of intelligent mines. In this paper, an image-based weight estimation system of coal and gangue on conveyor belt is established by combining deep learning-based case segmentation algorithm and linear regression model. The experimental results show that the whole system has good stability and the measurement accuracy is 87%, which provides a new intelligent solution for coal production and utilization.