Real-time high-resolution three-dimensional (3D) reconstruction of scenes hidden from the direct field of view is a challenging field of research, with applications in real-life situations related, e.g., to surveillance, self-driving cars, and rescue missions. Most current techniques recover the 3D structure of a non-line-of-sight (NLOS) static scene by detecting the return signal from the hidden object on a scattering observation area. Here, we demonstrate the full color retrieval of the 3D shape of a hidden scene by coupling back-projection imaging algorithms with the high-resolution time-of-flight information provided by a single-pixel camera. By using a high-efficiency single-photon avalanche-diode (SPAD) detector, this technique provides the advantage of imaging with no mechanical scanning parts, with acquisition times down to the subsecond range.