Resource provisioning aims to efficiently allocate the infrastructure provider's communication, computation and caching resources to provide reliable (e.g., 99.999%) and responsive actions for massive delay-critical services. However, the challenges arise from the partial knowledge of uncertain traffic arrivals (even the distributions of estimation error may not be available) and computation complexity for massive numbers of emerging services. This paper aims to design efficient and robust resource provisioning for massive delay-critical services based on only partial knowledge on traffic uncertainty. We formulate the problem of robust resource provisioning to minimize the system cost with only partial (i.e., the first/second momentum) information of traffic estimation errors. We derive the robust approximation of the reliability constraint using the Bernstein approximation, which is proved to guarantee service reliability given the partial traffic knowledge. The problem is reformulated and solved via Lagrangian duality to obtain the closed-form expression for low-complexity solutions. Experimental results on both the simulation-based and trace-based datasets validate that the proposed approach can guarantee up to 99.999% service reliability with reduced time complexity.