The integration of inertial navigation system (INS) and global navigation satellite system (GNSS) is considered as a promising device to recover the gravity vector, including deflections of the vertical (DOV) and gravity anomaly. However, the coupling between the DOV and the horizontal attitude errors will induce significate measurement errors of DOV. In order to improve the accuracy of the DOV measurement, a new algorithm based on the observation of the rotational inertial navigation system (RINS) and GNSS integration attitude errors is proposed. Firstly, an attitude reference decoupled from DOV is constructed with the raw gyroscope data of the INS. Then, the attitude error of the RINS/GNSS integration, which is directly induced by the DOV signal, is obtained when compared with the attitude reference. Thirdly, DOV and various systematic errors including attitude reference error and inertial sensor errors are accurately modeled. The long-wavelength components of DOV are given by global gravity model, and the short-wavelength components are modeled as derivative second-order Gauss-Markov processes. Finally, DOV and systematic errors are simultaneously estimated by a Kalman smoother with the observation of RINS/GNSS attitude error. A ship-borne survey campaign deploying three RINS/GNSS integrated systems is conducted to validate the feasibility of the proposed method. The experimental results show that the internal accuracy of the DOV measurement can reach better than 0.25″(1σ). When wavenumber correlation filter (WCF) algorithm is applied to the DOV data obtained by each system along the repeated lines, the internal accuracy can be improved to 0.1″(1σ).