Neural Radiance Field (NeRF) is a state-of-the-art algorithm in the field of novel view synthesis and has the potential to be used in AR/VR. However, the inference of NeRF is time-consuming. Motivated by resource-constraint scenarios on the edge and mixed reality devices, our essential idea is to bridge this gap while improving throughput and power consumption. This paper proposes a high-performance FPGA-based accelerator, with a fully-pipelined design tailored for the vanilla NeRF algorithm. We also design a mechanism to monitor the output of the rendering module to reduce operations. Experimental results show that our accelerator achieves 3.63× energy efficiency over implementation on GPU NVIDIA V100, and 1.31× speed up over state-of-the-art ASIC design if running under the same clock frequency as ASIC.