Synonym discovery, also known as synonym relation mining or synonym extraction, aims to identify and establish synonymous relationships between words, phrases, or sentences. The primary objective of this relationship mining is to enhance the performance of natural language processing (NLP) tasks, such as information retrieval, question-answering systems, text summarization, and machine translation. Considering that there are still numerous areas and issues awaiting further research in synonym relation mining, this paper provides a comprehensive review of the research methods employed in this field over the past two decades. The review is organized into four main categories. The first category explores the u e of language models for synonym extraction, including techniques such as word embeddings and the recent BERT model. The second category focuses on computing semantic similarity in synonym semantic spaces. The third category examines neural network-based approaches for synonym relation mining. Lastly, the fourth category delves into constructing synonym relationships based on knowledge graphs. Additionally, this paper provides an outlook and summary of potential future developments in this direction, with the aim of offering valuable guidance for future research in the field.