In this study, we present first-principles investigations of the atomic structure of Al1−xScxN and its influence on its piezoelectric and ferroelectric properties. The unbiased structure searching revealed that Al1−xScxN with phase separation feature, where AlN and ScN form a layered structure with different symmetries, is more stable than the corresponding wurtzite structure. The piezoelectric response of Al1−xScxN is strongly dependent on the atomic arrangements; in particular, Al0.5Sc0.5N with a wurtzite structure exhibits a large positive e33 of 4.79 C/m2, whereas Al0.5Sc0.5N with a phase separation structure exhibits a negative e33 of −0.67 C/m2. Moreover, the ferroelectric switching of Al1−xScxN demonstrated two distinct pathways for the wurtzite and phase separation structures, and the spontaneous polarization thus calculated exhibits entirely different values. Accordingly, we demonstrated that Al0.25Sc0.75N with a phase separation structure exhibits a low polarization switching barrier of 0.15 eV/f.u. and a large spontaneous polarization of −0.77 C/m2; thus, it can serve as a novel Al1−xScxN-based ferroelectric material. As the dipoles in Al1−xScxN with a phase separation structure are localized in the AlN region, they are individually switchable at no domain wall energy cost and are stable against extrinsic effects.