Speaker verification in household scenarios is a challenging problem due to the similarity in voice characteristics, such as accent, prosody, and intonations, due to genetic and environmental factors shared by household members. Therefore, a universal embedding space may not be optimal for household speaker verification. To solve this problem, we developed a distance metric for speaker verification in a household setting based on principal components of enrolled embedding vectors to adapt to the distribution of household members. Evaluated on simulated noisy-reverberation households from the VoxCeleb1 dataset, our approach reduces identification equal error rate reduction by 1.16% to 7.11% relatively.