This paper presents a modular and scalable machine learning framework for multi-material magnetic core loss modeling. The neural network framework is trained to predict core loss based on a flux density excitation waveform B(t) as well as additional scalar inputs including temperature, frequency, and dc-bias in order to handle a wide range of operating conditions. The framework is implemented such that a large portion of the model, the feature extractor, is shared for multiple materials, while specific materials require very few parameters in individual feature mapping networks. This allows the framework to 1) effectively model various materials with a scalable neural network structure and low parameter count; 2) accurately predict core losses across a wide operation range; and 3) adaptively support new materials with additional material-specific mapping networks trained with limited new data.