Acoustic phonons have recently been posited as playing an integral role in explaining the half-quantized thermal Hall effect in $\alpha$-RuCl$_3$. Therefore, we present much needed inelastic x-ray scattering measurements of its acoustic phonon dispersion, along with calculations using the frozen-phonon method. We also discuss a temperature study which conclusively shows a first-order structural transition to a non-$C2/m$ space group at low temperature. Together these results are an important backbone for future theoretical and experimental studies of $\alpha$-RuCl$_3$.