Artificial intelligence (AI) is revolutionizing the medical field in various ways. However, despite the successful implementation of various AI models in the healthcare field, the complexity, high dimensionality, and nonlinearity of AI hinder its widespread application due to the opacity of the models. The role of eXplainable artificial intelligence (XAI) is basically the same, providing clinicians and experts with predictive, diagnostic, and explanatory information about clinical decision-making patterns, and providing patients with explanations of the reasoning process and results generated by AI, thereby improving their acceptance and trust in such explanations.