Since thermal fluctuations become more important as dimensions shrink, it is expected that low-dimensional magnets are more sensitive to lattice distortions and phonons than bulk systems are. Here we present a fully relativistic first-principles study on the spin-lattice coupling, i.e. how the magnetic interactions depend on local lattice distortions, of the prototypical two-dimensional ferromagnet CrI$_3$. We extract an effective measure of the spin-lattice coupling in CrI$_3$ which is up to ten times larger than what is found for bcc Fe. The magnetic exchange interactions, including Heisenberg and relativistic Dzyaloshinskii-Moriya interactions, are sensitive both to the in-plane motion of Cr atoms and out-of-plane motion of ligand atoms. We find that significant magnetic pair interactions change sign from ferromagnetic (FM) to anti-ferromagnetic (AFM) for atomic displacements larger than 0.16 {\AA}. We explain the observed strong spin-lattice coupling by analyzing the orbital decomposition of isotropic exchange interactions, involving different crystal-field-split Cr$-3d$ orbitals. The competition between the AFM t$_{2g}$ - t$_{2g}$ and FM t$_{2g}$ - e$_{g}$ contributions depends on the bond angle formed by Cr and I atoms as well as Cr-Cr distance. In particular, if a Cr atom is displaced, the FM-AFM sign change when the I-Cr-I bond angle approaches 90$^\circ$. The obtained spin-lattice coupling constants, along with the microscopic orbital analysis can act as a guiding principle for further studies of the thermodynamic properties and combined magnon-phonon excitations in two-dimensional magnets.