A facile electrospinning technique with subsequent pyrolysis is employed to prepare porous carbon nanofibers containing uniformly dispersed ultrasmall Sb nanoparticles (denoted as Sb/CNF). Being the Sb source as well as pore forming agent, potassium antimony tartrate is crucial to construct mesopores in CNF. This useful design provides distinct advantages for Sb-based anodes in both sodium-ion batteries and potassium-ion batteries. As a sodium-ion battery anode, the porous Sb/CNF-0.5 composite exhibits a high reversible capacity of 462 mA h g−1 at 100 mA g−1 after 100 cycles. High cyclic stability with 90.6% capacity retention after 500 cycles at 500 mA g−1 was achieved for the optimized electrode. For potassium storage, Sb/CNF-0.5 exhibits a high capacity of 450 mA h g−1 over 100 cycles at 100 mA g−1, and a stable capacity of 403 mA h g−1 over 500 cycles at 500 mA g−1. These superior electrochemical properties are attributed to the unique structure of porous nanofibers confining ultrasmall Sb nanoparticles. Porous CNF not only functions as a conductive support to improve the electrical conductivity of the electrode, but also provides a buffering matrix for releasing the mechanical stress caused by Na+/K+ insertion/extraction efficiently. Furthermore, according to molecular dynamics simulations, the optimized mesoporous structure in CNF can offer a more facile access to the electrolyte comparing to the microporous structure, thus improving the electrochemical energy storage performance.