In this paper, we first establish the separation theorem between a point and a locally geodesic convex set and then prove the existence of a supporting quasi-hyperplane at any point on the boundary of the closed locally geodesic convex set on a Riemannian manifold. As applications, some optimality conditions are obtained for optimization problems with constraints on Riemannian manifolds.