Hyperspectral image has been widely used in the field of remote sensing due to the rich spectral and spatial information. During land-cover investigation, new types of ground objects appear constantly, which need the classification model to make quick judgments. However, a trained hyperspectral classification model can only make predictions on pre-defined classes, which limits its application. In order to deal with the aforementioned issue, we propose an incremental learning method based on constantly updated classifier, which is able to recognize new classes by few-shot samples and classify old classes without storing any old samples. Specifically, we propose a decoupled structure of feature representation module and classifier module, the feature representation is trained in the initial stage and then frozen, while the classifier module is updated in the next incremental stages, which alleviates knowledge forgetting as well as over-fitting. Besides, we adopt attention mechanism with a kernel of cosine distance to measure the similarity between prototypes and test samples of each class, which is more robust with few-shot samples. Extensive experiments and analyses based on typical hyperspectral images verified the effectiveness of the proposed method.