Image enhancement is an essential processing for raw infrared images to improve contrast. In this paper, an infrared image enhancement method is proposed to solve the problem of over-enhancement on background in occurrence probability histogram based methods and the problem of uneven enhancement for targets in the saliency weight based method. First, the saliency histogram is constructed, where the bin values of target-related intensity levels are significantly larger than the bin values of background-related intensity levels. According to the value distribution of bins in saliency histogram, the intensity levels with several largest bin values, defined as the salient intensity levels, are segmented. Salient intensity levels are separated further away from non-salient levels to improve the contrast between targets and background. Second, salient intensity levels are divided into groups and each group is related to targets of a certain range of temperatures. Bin values of salient intensity levels are reallocated within each group to keep these intensity levels compact, for the purpose of performing equalized enhancement for targets. Experimental results demonstrate that the proposed method achieves better improvement on target-to-background contrast and better visual quality on targets by performing equalized enhancement for targets.