We have investigated the structural changes and the microstructures of Ba1−xSrxAl2O4 for 0 < x < 0.4 by using transmission electron microscope (TEM) and synchrotron radiation powder X-ray diffraction experiments. The TEM experiments revealed the existence of a structural phase boundary at approximately x = 0.1, at which the superlattice reflection spots at the 1/2 0 0 −type positions change into diffuse streaks along three equivalent <110> directions in the hexagonal structure. In addition, real-space images of Ba1−xSrxAl2O4 for 0 < x < 0.4 reveal that BaAl2O4 should be characterized as a modulated structure with triple-q modulation vectors along the <100> directions and on the other hand, Ba1−xSrxAl2O4 for 0.1 < x < 0.4 be characterized as an intermediate (precursor) state with a rigid unit mode due to structural instability. These experimental results implied that the partial substitution of Sr2+ for Ba2+ should suppress a structural instability due to the AlO4 tetrahedral network and decrease the structural phase transition temperature.