Lung motion due to respiration causes image degradation in medical imaging, especially in nuclear medicine which requires long acquisition time. We have developed a method for image correction between the respiration-gated (RG) PET images in different respiration phases or breath-hold (BH) PET images in inconsistent respiration phase. In the method, the RG or BH PET images in different respiration phase are deformed under two criteria; similarity of image density distribution and smoothness of estimated motion vector field (MVF). However, only these criteria may cause un-natural motion estimation of lung. In this paper, assuming the use of a PET-CT scanner, we add another criterion that is the similarity to the motion direction estimated from inhalation and exhalation CT images. The proposed method was applied to XCAT phantom image data and seven patients' BH-PET image data. Successful registration results were obtained by the proposed method.