Breast cancer ranks as utmost prevalent form of cancer among women on a global scale, and a pathologist must spend a lot of time studying many images under various magnifications in order to provide an accurate and prompt diagnosis. Many researchers automate this procedure in order to provide a quicker and more reliable diagnosis of these malignancies. They do this by using computer vision, machine learning, and deep learning approaches. This paper is meant for the various techniques and algorithms used for the detection of cancerous nuclei. There are the variations in shape, size and appearance and texture in the nuclei of breast cancer in histopathology images, there is a need for the automated nuclei detection. And it is the difficult field in the computer aided pathology research. In this study, For the automated identification of the nuclei of breast cancer histopathology images, Mask RCNN (Region-Based Convolution Neural Network) method is used. Various methods like Feature Pyramid Networks(FPN), Fully Convolution Networks(FCN), ROIAlign and ResNet network are used by Mask RCNN. ROIAlign is used as it increases accuracy of the nuclei detection. FCN helps to predict the results in more detail. The study also reflects that Mask RCNN algorithm is more superior compared to the other nuclei detection algorithms.