In this paper we propose a methodology of multilayer filters based on tempo variation for realizing a query by humming (QBH) system. Firstly the original query clip is used to search for the candidate songs. If the results are unreliable, the clip is linearly scaled twice for more candidates. If the results are still unreliable, the clip is scaled more times for retrieval. To sort all the candidates, a new matching algorithm called key transposition recursive alignment (KTRA) is presented, which improves the retrieval accuracy. Experimental results on the 2010 MIREX QBH query corpus show that the proposed method can achieve a relative improvement of 20.9% as well as an acceleration factor of 2.09 simultaneously compared to a state-of-the-art method.