The magnetically guided localization technique is a promising modality for the breast-conserving cancer treatment, in which a magnetic marker buried in the cancer is detected using the magnetic probe. Recently, a least square-based estimation method of the marker has been proposed to achieve more accurate localization than simply exploring the direction that maximizes the measured magnetic field. We previously proposed a handheld magnetic probe equipped with a permanent magnet to magnetize the marker and enhance its magnetic moment and validated the high detection rate through clinical tests. In this article, we propose a novel least square-based localization method that is applicable to our handheld magnetic probe. Since the magnetic moment of the marker varies moment-by-moment due to the magnetic field generated by the permanent magnet of the probe, a magnetization model must be incorporated in the estimation. We adopt the play model and combine it with the nonlinear least squares method to develop the estimation algorithm. We validate the proposed method by experiments and it is demonstrated that the proposed method can accurately and precisely estimate the depth of the marker under a constantly magnetized environment.