Monolayer semiconducting transition metal dichalcogenides possess broken inversion symmetry and strong spin-orbit coupling, which leads to unique spin-valley locking effect. In 2H stacked pristine multilayers, the spin-valley locking yields an electronic superlattice structure, where alternating layers correspond to barrier and quantum well respectively, conditioned on the spin-valley indices. Here, we show that the spin-valley locked superlattice hosts a new kind of dipolar excitons with the electron and hole constituents separated in an every-other-layer configuration, i.e., either in two even or two odd layers. Such excitons become optically bright via hybridization with intralayer excitons, displaying multiple anti-crossing patterns in optical reflection spectrum as the dipolar exciton is tuned through the intralayer resonance by electric field. The reflectance spectra also reveal an excited state orbital of the every-other-layer exciton, pointing to a sizable binding energy in the same order of magnitude as the intralayer exciton. As layer thickness increases, the dipolar exciton can form one-dimensional Bose-Hubbard chain displaying a layer number dependent fine-structures in the reflectance spectra. Our work reveals a distinct valleytronic superlattice with highly tunable dipolar excitons for exploring light-matter interactions.