This paper investigates the nonlinear control problem for hypersonic vehicle with uncertainties. The model of hypersonic vehicle is established. Considering the strong nonlinearity and strong coupling of hypersonic vehicle longitudinal model, the dynamic inversion is applied to decouple it. To overcome undesirable response caused by unmolded dynamics and external disturbances, the extended state observer is developed to estimate and compensate them. Meanwhile, the parameters of observer are optimized by using particle swarm optimization algorithm to improve the performance of controller. Numerical simulation is given to verify the effectiveness of proposed control method and PSO algorithm, where the unmolded dynamics and external disturbances is considered. The results of simulation show that the controller designed by proposed method have good tracking performance.