Electron-phonon coupling, i.e., the scattering of lattice vibrations by electrons and vice versa, is ubiquitous in solids and can lead to emergent ground states such as superconductivity and charge-density wave order. Strong coupling of phonons to electrons near the Fermi surface, which reduces the phonon lifetimes and broadens the phonon peaks in scattering experiments, is often associated with Fermi surface nesting. Here, we show that strong phonon broadening can occur in the absence of both Fermi surface nesting and lattice anharmonicity, if electron-phonon coupling is strongly enhanced for specific values of electron-momentum, k. We use inelastic neutron scattering, soft x-ray angle-resolved photoemission spectroscopy measurements and ab-initio lattice dynamical and electronic band structure calculations to demonstrate this scenario in the highly anisotropic tetragonal electron-phonon superconductor YNi$_2$B$_2$C. This new scenario likely applies to a wide range of compounds.