U-Net vs HoVer-Net: A Comparative Study of Deep Learning Models for Cell Nuclei Segmentation and Classification in Breast Cancer Diagnosis

Abstract

Breast cancer is a prevalent form of cancer and a leading cause of cancer-related deaths in women worldwide. Early detection and accurate diagnosis are crucial for effective treatment and improved patient outcomes. Pathologists play a critical role in breast cancer diagnosis by examining tissue samples under a microscope, but this process can be time-consuming and prone to errors. Deep learning techniques have demonstrated potential in automating this process and reducing the workload of pathologists. The primary objective of this thesis is to compare the performance of two algorithms based on the U-Net and HoVer-Net architectures for the task of cell nuclei segmentation and classification of immunohistochemistry images in the context of breast cancer diagnosis. Additionally, we investigate the feasibility of developing a hybrid algorithm that combines the best features of both architectures. The models were trained using three different types of staining, that specifically target the cell nucleus: Ki-67, ER, and PR. The results showed that HoVer-Net outperformed U-Net for the Ki-67 dataset. However, a hybrid algorithm, combining the pixel-level cell classification of U-Net and the cell centroids of HoVer-Net, achieved the highest cell-level F-score for the ER and PR datasets.