Filling layer self-compacting concrete (FLSCC) is an essential material used in the construction of China Rail Track System type III (CRTS III) slab track of high-speed rail. The mechanical properties of the filling layer are of paramount significance to the performance and serviceability of the track system. In this study, asphalt emulsion (AE) was incorporated into the FLSCC, and various properties related to workability, strength, durability, and damping performance were experimentally evaluated. The effect of AE on the durability of FLSCC was investigated through water absorption, sorptivity, and rapid chloride ion penetration tests; while damping properties were evaluated using impact resonance method. Results show that addition of AE decreases the compressive and split tensile strength while flexural strength increases, especially at lower AE content. The durability properties were greatly enhanced with the addition of AE, which results from the decreased permeability of FLSCC specimens due to the pore-filling effect of asphalt film that flows and fills the pores and cracks of the matrix. Damping ratio of FLSCC increased by 22.8%, 23.7%, 25%, and 30.6% through the addition of 5%, 10%, 15%, and 20% AE, respectively. According to the findings of this study, AE content up to 10% is recommended for FLSCC production.