The constraint following control problem of the mechanical systems is investigated in this paper. For the practical mechanical system, there always exists the time-varying uncertainty, which may be due to the system parameter perturbation, the modeling error, and the external disturbance. The uncertainty will result in the constraint following performance degradation. To solve this issue, a new adaptive robust control is proposed to force the system to satisfy the constraint. The control scheme consists of two parts: the nominal part and the adaptive robust part. The nominal part is proposed based on the Udwadia-Kalaba theory, which can guarantee the constraint following performance of the nominal system. The adaptive robust part is proposed to compensate for the uncertainty. In addition, the adaptive law is proposed to estimate the uncertainty bound of the system. The resulting adaptive robust control can guarantee the uniform boundedness and uniform ultimate boundedness of the constraint following system in the presence of various uncertainties, which is verified by a two-link Revolute-Prismatic Mechanical Manipulator (RPMM) system.