Mannosyl compounds have been widely used in nutrition, fodder, and vaccine adjuvant industries. In our previous study, the engineered strains for the biosynthesis of three mannosyl compounds including β‐1,2‐mannobiose (M2‐β‐1,2), β‐1,2‐mannotriose (M3‐β‐1,2), and mannosylglycerate (MG) have been developed. However, their biological activities have not been reported. Here, those three compounds were successfully purified after fermentation of the engineered strains, and their potential immunomodulatory activities on RAW264.7 macrophages were investigated with commercialized β‐1,4‐mannotriose (M3‐β‐1,4) as control. Our results showed that M3‐β‐1,2 and MG promoted the viability and phagocytic function of RAW264.7. Meanwhile, the cytokine TNF‐α and interleukin‐6 (IL‐6) level of RAW264.7 macrophages were significantly enhanced upon the stimulation of M3‐β‐1,2 and MG compared with M3‐β‐1,4. Moreover, MG significantly stimulated macrophages to secrete IL‐10 compared with other mannan oligosaccharides. Finally, this study proved that the immunomodulatory activity of M3‐β‐1,2 and MG on RAW 264.7 cells was mainly through mitogen‐activated protein kinases and myeloid differentiation protein 88 (MyD88)‐dependent signaling pathways. All these findings suggested that M3‐β‐1,2 and MG exhibited immunomodulatory activities in the innate and adaptive immune systems, thus facilitating the application potential in developing of mannosyl compounds as an immunomodulator available for food and pharmaceutical area. (a) Mannosyl compounds obtained from microbial fermentation exhibited immunomodulatory activities in the innate and adaptive immune systems. (b) The immunomodulatory activity of mannosyl compounds on RAW 264.7 cells mainly functioned through mitogen‐activated protein kinases and MyD88‐dependent signaling pathways. (c) The length of the mannosyl chain and the type of glycosidic bounds influence the function of mannan oligosaccharides.