Lithium ion (Li+) battery with conventional liquid electrolyte has typical and notorious security risk. Composite solid electrolyte integrating the merits of various electrolyte systems is an attractive alternative of liquid electrolyte for new-type safety and long-life lithium (Li) batteries. Herein, a 3D framework composite solid electrolyte based on Al2O3-modified Li1.4Al0.4Ti1.6(PO4)3 (Al2O3@LATP) was prepared by solution casting and scraping method. The 3D framework composite electrolyte exhibits admirable ionic conductivity (1.1 × 10−4 S cm−1 at 60 °C), wide voltage window (4.8 V vs. Li/Li+), good thermal stability, and satisfactory electrolyte/electrode interfacial compatibility. The Al2O3@LATP can not only impede Li dendrite growth, but also avoid the LATP reaction with Li anode. The assembled solid-state LiFePO4|Li battery with 3D framework composite solid electrolyte delivers a satisfactory capacity retention up to 95.2% after 400 cycles at 0.5 C and under 60 °C. The 3D framework structural design of solid electrolyte affords a promising option for realizing safe energy storage systems.