Accretion-powered millisecond X-ray pulsations have been detected in only 13 of more than 130 known neutron stars in low-mass X-ray binary systems (LMXBs). Pulsations have been detected only intermittently in three of these accretion-p owered millisecond X-ray pulsars. We show that the failure to detect accretion-powered pulsatio ns from most neutron-star LMXBs, and the intermittent detection of pulsations in a few, can both b e explained if the emitting regions of accreting neutron stars with millisecond spin periods are t ypically close to their spin poles and move in response to changes in the accretion flow. This nearly -aligned moving spot model is consistent with the small observed amplitudes and nearly sinusoidal waveforms of most accreting millisecond X-ray pulsars, and the large, rapid phase variations of several.