The layered honeycomb magnet $\alpha$-RuCl$_3$ has been suggested to exhibit a field-induced quantum spin liquid state, in which the reported large thermal Hall effect close to the half-quantized value still remains a subject of debate. Recently, oscillatory structures of the magnetothermal conductivity were reported and interpreted as quantum oscillations of charge-neutral particles. To investigate the origin of these oscillatory structures, we performed a comprehensive measurement of the in-plane magnetothermal conductivity $\kappa(H)$ down to low temperature (100 mK), as well as magnetization $M$, for single crystals grown by two different techniques: Bridgman and chemical vapor transport. The results show a series of dips in $\kappa(H)$ and peaks in the field derivative of $M$ located at the same fields independent of the growth method. We argue that these structures originate from field-induced phase transitions rather than quantum oscillations. The positions of several of these features are temperature-dependent and connected to the magnetic phase transitions in zero field: the main transition at 7 K and weaker additional transitions which likely arise from secondary phases at 10 K and 13 K. In contrast to what is expected for quantum oscillations, the magnitude of the structure in $\kappa(H)$ is smaller for the higher conductivity crystal and decreases rapidly upon cooling below 1 K.