We reported proton-conducting membranes with novel microstructure based on partially phosphorylated poly(vinyl alcohol) (P-PVA) and polyethylene glycol (PEG) grafted silica (PEG-SiO2) particles. The PEG-SiO2 particles were synthesized through acid catalyzed hydrolysis and condensation reactions. The membranes were characterized for their mechanical, structural, morphological, and electrical properties by employing tensile test, Fourier transform infrared (FTIR), scanning electron microscopy (SEM), thermogravimetric analysis (TGA), differential scanning calorimetry (DSC), impedance analyzer, respectively. In these membranes, P-PVA acts as the proton source and PEG act as the proton solvent. The PEG-riched phases in the hybrid membrane form continuous ionic conducting pathways and subsequently give high ionic conductivity. The results suggest that the obtained membrane shows good thermal stability, excellent mechanical property and high ionic conductivity, and the low-cost hybrid membrane can be a promising candidate for intermediate temperature fuel cell systems.