Predicting financial stock prices, which are complex, volatile, and nonlinear, poses a significant challenge due to the multitude of influencing factors and inherent uncertainty in the financial market. This paper introduces a novel approach that utilizes a neural network model combining the Hodrick-Prescott (HP) filter and a Multi-Scale Gaussian transformer to tackle these challenges. The proposed method enhances the model’s local features by incorporating a Multi-Scale Gaussian transformer. Initially, the stock’s time series is decomposed into long-term and short-term fluctuations using the HP filter. Subsequently, the encoded long-term and short-term series are fed into a Multi-Scale Gaussian transformer. Additionally, a Multi-Scale Gaussian prior is introduced to further boost the local features of the transformer and enhance the relative positional information features of the time series. In comparison to popular recurrent neural networks like RNN, LSTM, GRU, and state-of-the-art baseline models, our model (HPMG-Transformer) offers a unique advantage in capturing both extremely long-term and short-term dependencies in stock time series. Experimental results illustrate the significant benefits of our proposed model in predicting stock trends in the China A-shares market, New York Stock Exchange (NYSE), and NASDAQ market.