Breast cancer molecular subtyping from H&E whole slide images using foundation models and transformers

Abstract

This study tackles the challenge of classifying breast cancer molecular subtypes using H&E-stained Whole Slide Images (WSI), avoiding the cost and labor limitations of the commonly used immunohistochemistry. We leverage the Attention-Challenging Multiple Instance Learning framework and introduce a variant, ACTrans, which utilizes a transformer aggregator for a more flexible feature aggregation. We also compare two publicly available foundation feature extractors pre-trained on large pathology datasets. A comparison of the impact of two different patch sizes at two different resolutions is made. The results obtained in our in-house dataset demonstrate that ACTrans yields superior results than existing methods, particularly with the UNI model at lower resolutions and larger patch sizes. In this setting, ACTrans achieves an average F1 score of 0.687, a precision of 0.755, a recall of 0.667, and an AUC of 0.812. Furthermore, these approaches enhance interpretability when displaying the attention weights. This method can potentially advance breast cancer diagnostics by leveraging the rich information within H&E-stained WSIs.