In this study, flake graphite is used as raw material and oxidized to produce graphene oxide (GO), and then two kinds of ferroferric oxide/reduced graphene oxide (Fe3O4/rGO) composites are fabricated with one-step and two-step methods, respectively. The influence of composite methods on structure and performance is investigated by FT-IR, XRD, SEM, TEM, and testing of electromagnetic parameters. The results show that there are many advantages in the one-step method for fabrication of Fe3O4/rGO composite, such as the composite process is simplified, the time is shortened, and the Fe3O4/rGO composite fabricated with the one-step method has the best microwave impedance matching property in the frequency range of 2.5–6 GHz. At the same time, it can be found that there is the Fe–O–C bond in Fe3O4/rGO composites fabricated with the one-step method, which results in two ways of physical recombination and chemical bonding between Fe3O4 and rGO sheet particulates. In addition, when the mass ratio of GO and FeCl3 is 1:10, the RL value of the Fe3O4/rGO composite fabricated with the one-step method is minimum, and the increase of sample thickness is beneficial to improve the impedance matching property of Fe3O4/rGO composite in the frequency range of 2.5–6 GHz.