How to exploit the rich information contained in Polarimetric Synthetic Aperture Radar (PolSAR) data, has recently gained much attention for PolSAR image interpretation via Deep Learning. In this paper, a polarimetric phase difference aided approach is presented for PolSAR image classification. The polarimetric phase differences conveyed by the off-diagonal elements, as well as the diagonal components in the coherence matrix, are extracted to form a 6-D target vector, i.e., the input to a deep model includes 6 channels. The experimental results on benchmark PolSAR data indicate that the polarimetric phase difference is indeed information bearing, moreover, the proposed strategy can be flexibly implemented by current deep learning framework without modification.