We report a thermal transport study of IrSbSe, which crystallizes in a noncentrosymmetric cubic structure with the $P2_13$ space group and shows a narrow-gap semiconducting behavior. The large discrepancy between the activation energy for conductivity [$E_\rho$ = 128(2) meV] and for thermopower [$E_S$ = 17.7(9) meV] from 200 to 300 K indicates the polaronic transport mechanism. Electrical resistivity varies as $exp(T_0/T)^{1/4}$ and thermopower varies as $T^{1/2}$ at low temperatures, indicating that it evolves into the Mott's variable-range hopping dominant conduction with decreasing temperature. IrSbSe shows relatively low value of thermal conductivity ($\sim$ 1.65 W/K$\cdot$m) and thermopower of about 0.24 mV/K around 100 K, yet poor electrical conductivity. On the other hand, high vacancy defect concentration on both Ir and Sb atomic sites of up to 15\%, suggests high defect tolerance and points to possibility of future improvement of carrier density by chemical substitution or defect optimization.