This paper describes the composites of silane cross-linked polyethylene (PEXb) mixed with four different fillers dedicated to improving thermal conductivity, such as multi-walled carbon nanotubes, graphite, boron nitride, and mineral filler. Using the mathematical modeling, the concentration of each filler was estimated to reach the value of 0.675 W m−1 K−1 suitable for the usage PEXb in the production of geothermal pipes. From tested fillers, all improve the thermal conductivity of pure PEXb; however, above the required value, only 28 mass% of boron nitride (0.937 W m−1 K−1). It was associated with its perfect dispersion and the highest gel content achieved, also resulting in the highest flexural modulus. Graphite at 5 mass% and mineral filler at 35 mass% allowed the thermal conductivity to equal 0.622 W m−1 K−1 and 0.624 W m−1 K−1, respectively, with a simultaneous medium effect on the mechanical properties. Carbon nanotubes increased PEX's conductivity to a small extent, due to their occurrence as agglomerates.