The recent report of a half-quantized thermal Hall effect in the Kitaev material $\alpha$-RuCl$_3$ has sparked a strong debate on whether it is generated by Majorana fermion edge currents or whether other more conventional mechanisms involving magnons or phonons are at its origin. A more direct evidence for Majorana fermions which could be expected to arise from a contribution to the longitudinal heat conductivity $\kappa_{xx}$ at $T\rightarrow0$ is elusive due to a very complex magnetic field dependence of $\kappa_{xx}$. Here, we report very low temperature (below 1~K) thermal conductivity ($\kappa$) of another candidate Kitaev material, Na$_2$Co$_2$TeO$_6$. The application of a magnetic field along different principal axes of the crystal reveals a strong directional-dependent magnetic-field ($\bf B$) impact on $\kappa$. We show that no evidence for mobile quasiparticles except phonons can be concluded at any field from 0~T to the field polarized state. In particular, severely scattered phonon transport is observed across the $B-T$ phase diagram, which is attributed to prominent magnetic fluctuations. Cascades of phase transitions are uncovered for all $\bf B$ directions by probing the strength of magnetic fluctuations via a precise record of $\kappa$($B$). Our results thus rule out recent proposals for itinerant magnetic excitations in Na$_2$Co$_2$TeO$_6$, and emphasise the importance of discriminating true spin liquid transport properties from scattered phonons in candidate materials.