We present a comprehensive study of the non-centrosymmetric semimetal LaRhGe$_3$. Our transport measurements reveal evidence for electron-hole compensation at low temperatures, resulting in a large magnetoresistance of 3000% at 1.8 K and 14 T. The carrier concentration is on the order of $10^{21}\rm{/cm}^3$, higher than typical semimetals. We predict theoretically the existence of $\textit{almost movable}$ Weyl nodal lines that are protected by the tetragonal space group. We discover superconductivity for the first time in this compound with a $T_{\text c}$ of 0.39(1) K and $B_{\rm{c}}(0)$ of 2.1(1) mT, with evidence from specific heat and transverse-field muon spin relaxation ($\mu \rm{SR}$). LaRhGe$_3$ is a weakly-coupled type-I superconductor, and we find no evidence for time-reversal symmetry breaking in our zero-field $\mu \rm{SR}$. We study the electrical transport in the normal state and find an unusual $\sim T^{3}$ dependence at low temperature while Seebeck coefficient and thermal conductivity measurements reveal a peak in the same temperature range. We conclude that the transport properties of LaRhGe$_3$ in its normal state are strongly influenced by electron-phonon interactions. Furthermore, we examine the temperature dependent Raman spectra of LaRhGe$_3$ and find that the lifetime of the lowest energy $A_1$ phonon is dominated by phonon-electron scattering instead of anharmonic decay.