The central problem in hyperspectral remote sensing is characterizing the material components of a scene based on the spectral radiance observed in the image pixels. What makes this challenging is the fact that the spectral response for a given material exhibits considerable variability from a variety of causes: intrinsic (depending on the composition or morphology of the material), extrinsic (depending on the size of an object or the concentration of the material), or environmental (due to illumination, atmospheric distortion, and so on). In this article, we survey many of the causes of spectral variability, describe spectral models for this variability, and outline some signal processing and target detection strategies for analyzing hyperspectral data in a way that is more robust to this variability.