The boron-lined gaseous neutron detector has been shown as a promising detector that may replace the $^{\mathbf {3}}$He counter. The roughness of the boron layer, which is one of the key components in this detector, can affect the probabilities of both neutron absorption and charged particles escaping into the working gas, and in turn, have an impact on the detection efficiency of this detector. A research is conducted in this paper to investigate the relationship between the roughness of boron layer and the detector's performance. GEANT4 based simulations are carried to calculate the neutron absorption probabilities, charged particles escaping probabilities, detection efficiencies, and average energies of escaped charged particles versus different roughness of boron layer. Results show that a roughness smaller than 1 $\mu \textbf{m}$ has no appreciable effect on the detection efficiency. Thus, a newly developed boron coating method, dip-coating method that can realize boron layer roughness with the standard deviation of 0.38 $\mu \textbf{m}$, is acceptable for the boron layer production to manufacture the boron-lined gaseous neutron detector.