In this paper, we investigate joint caching and computing resource reservation for supporting location-aware augmented reality (AR) applications in an edge-assisted two-tier radio access network. We aim at minimizing the caching and computing resource consumption while satisfying the AR service delay requirement. Specifically, to capture the spatio-temporal AR service dynamics, the resource consumption minimization problem is formulated as a long-term stochastic optimization problem. Due to the time-varying service demands and tightly coupled multi-resource reservation decisions, we propose a novel resource reservation algorithm based on the Lyapunov optimization technique to solve the problem. We first transform the original long-term problem into multiple one-shot optimization problems, each of which is then solved by our designed iterative algorithm in an online manner. Simulation results demonstrate that the proposed algorithm can significantly reduce the overall resource consumption compared to benchmark algorithms.