We measure the complete set of angular coefficients $J_i$ for exclusive $\bar{B} \to D^* \ell \bar{\nu}_\ell$ decays ($\ell = e, \mu$). Our analysis uses the full $711\,\mathrm{fb}^{-1}$ Belle data set with hadronic tag-side reconstruction. The results allow us to extract the form factors describing the $B \to D^*$ transition and the Cabibbo-Kobayashi-Maskawa matrix element $|V_{\rm cb}|$. Using recent lattice QCD calculations for the hadronic form factors, we find $|V_{\rm cb}| = (41.0 \pm 0.7) \times 10^3 $ using the BGL parameterization, compatible with determinations from inclusive semileptonic decays. We search for lepton flavor universality violation as a function of the hadronic recoil parameter $w$, and investigate the differences of the electron and muon angular distributions. We find no deviation from Standard Model expectations.