Breast cancer is a heterogeneous disease characterized by abnormal proliferation of cells within the breast, exhibiting invasive growth, metastatic potential, and potential for recurrence. As of 2020, breast cancer has surpassed lung cancer to become the leading cause of cancer incidence globally, accounting for 11.7% of all cancer cases. In most countries, breast cancer has the highest incidence rate and the highest mortality rate among women in 110 out of 159 countries (Sung et al., 2021). Invasive breast cancer (IBC) is the most common type of breast cancer, making up about 75% of all cases. It is a malignant tumor in which cancer cells have invaded the basement membrane of the ductal or lobular follicles of the breast and the surrounding tissue. There are several subtypes of IBC, but the two most common are invasive ductal carcinoma (IDC) and invasive lobular carcinoma (ILC), with IDC accounting for up to 80% of cases. Grading IBC has been shown to have a significant impact on predicting clinical outcomes (Rakha et al., 2010).

The Nottingham Grading System (NGS), which employs a semi-quantitative approach, is commonly used for histological grading of IBC. This system assigns scores based on tubule formation, nuclear pleomorphism, and mitotic counts. These scores are then added together to classify IBC into three grades: Grade I, Grade II, and Grade III. Grade I IBC tends to grow slowly with limited spreading, while Grade III IBC grows more rapidly and has a higher degree of dissemination. Therefore, it is essential to make an accurate diagnosis of IBC as it enables more personalized treatment and prognosis assessment for patients.

Histopathology images of IBC exhibit high similarities across the three grades, making it a fine-grained classification problem. In particular, it is challenging to distinguish between Grade II and Grade I/III, as shown in Fig. 1. To address this challenge, it is necessary to develop an efficient and accurate algorithm for automatic identification of IBC images in different grades using artificial intelligence technology. Most of the current researches have focused on binary classification (i.e., non-malignant and malignant) (Roy et al., 2019, Ding et al., 2022), ternary classification (i.e., normal, benign, and malignant) (Zhang et al., 2023, Ting et al., 2019), and quaternary classification (i.e., normal, benign, in situ, and invasive carcinoma) (Roy et al., 2019, Bagchi et al., 2022), but only a few have investigated the tri-grading classification for the IBC diagnosis.

In general, these approaches can be broadly categorized into two main types: traditional methods and deep learning methods. Traditional methods use handcrafted features to characterize breast cancer cells with diverse shapes, textures, colors, and other attributes. The traditional approaches are favored due to their high interpretability and low data requirements, which can provide theoretical support and feasibility for clinical applications in breast cancer classification. For example, Kowal et al. (2013) extracted 42 morphological, topological, and textural features from segmented nuclei to classify breast cancer images. Gupta and Bhavsar (2017) extracted color and texture features along with an ensemble classifier framework to classify breast cancer histopathology images. Although traditional handcrafted feature extraction methods have achieved significant success in breast cancer classification, there is still room for improvement. These methods heavily rely on the subjective experience and domain knowledge of experts to select appropriate features, which can be prone to bias and incorrect feature selection. And the shortage of breast cancer pathologists also poses a big challenge for these methods.

To address the above issues, deep learning methods have become the mainstream approach for breast cancer classification due to their ability to automatically extract the most discriminative features from the original image data through end-to-end feature representations, leading to high-performance classification results. For example, Roy et al. (2019) utilized a convolutional neural network (CNN) to develop a patch-based classifier (PBC) for the automated binary classification and four-class classification on the ICIAR 2018 breast histopathology dataset. Yan et al. (2020) proposed a Nucleus-Aware Network (NANet) for grading IDC, which includes a nuclei branch to guide the model in extracting nuclei-related features during training. Ukwuoma et al. (2022) proposed “DEEP_Pachi”, a method that combines self-attention heads and multilinear perceptron to extract important information from regions of interest for classification on the BreakHis and ICIAR 2018 Challenge datasets. However, deep learning typically requires a large amount of annotated data for training, and the available publicly labeled IBC grade data is relatively scarce, which makes it difficult to construct and train deep learning models for IBC classification.

Considering the advantages and disadvantages of traditional methods and deep learning, it is necessary to combine them in order to make full use of both traditional and deep learning features to achieve complementary effects in breast cancer classification. For instance, Al-Jabbar et al. (2023) integrated CNN features (VGG-19 and ResNet-18) with handcrafted features to extract fusion features for classification on the BreakHis. However, there are still limited researches on the integrated approach for breast cancer histopathology image classification currently.

In the context of NGS analysis, there are actually multiple factors that can affect the diagnosis of IBC, including nuclear pleomorphism, mitotic count, tubule formation, and well-known epithelial region nuclei, all of which are closely related to IBC grading. Since the nuclei on different pathological regions exhibit varying quantities and spatial arrangements, the fusion of features extracted from multi sources holds the potential to improve the IBC diagnosis accuracy. Moreover, it is necessary to integrate more comprehensive pathology knowledge into the IBC diagnostic model and utilize complementary features extracted from multi-source domains for more effective grading of IBC.

In this paper, we propose a multi-source hybrid feature fusion network called MSFusion to fully integrate the complementary information among multi-source heterogeneous nuclei. Guided by the NGS closely related to IBC diagnosis and grading, we extract and fuse deep learning features and traditional handcrafted features from multiple hybrid sources. Specifically, the proposed network incorporates deep learning features extracted by a multi-Swin Transformer branch network called MSwinT, as well as traditional handcrafted features that describe the morphology of multi-source heterogeneous nuclei. The handcrafted features include Cell Architecture, Cell Morphology, Cell Orientation Entropy (COrE), Cell Cluster Graph (CCG), Haralick, and Quantitative Relations (QR). Besides, the primary branch of MSwinT is responsible for extracting the overall features of the original images, while multiple auxiliary branches are designed to identify morphological features from multi-source heterogeneous nuclei, including tumor, mitotic, tubular, and epithelial nuclei. At each stage in MSwinT, a functional KDC fusion block is introduced to integrate all the multi-source heterogeneous features. Finally, we synthesize the multi-source hybrid deep learning features and handcrafted features to improve the accuracy of IBC diagnosis and grading.

The main contributions of this paper are listed as follows:

A new idea to integrate multi-source pathological features guided by Nottingham grading system. Inspired by the actual diagnostic processes of clinicians and the internationally recognized “Nottingham” grading system, this study constructed a multi-branch MSwinT network based on the Swin Transformer architecture for extracting deep learning features. This network integrates multi-source pathological features and employs four important grading components as attention-guiding branches. This approach compels the network to take into account pathological information about the nuclei of tumor, mitotic, tubular, and epithelial regions, thereby enhancing the model’s discriminative power.

A novel multi-stage architecture to fuse raw pathological and key diagnostic components features. The MSwinT network consists of four stages, where multi-source pathological nuclear features are progressively integrated at different stages, layer by layer. The merged features serve as the input for the main branch in the next stage, rather than simply concatenating features at the final stage. This approach gradually enhances feature expression, retains rich detailed information, and optimizes the learning process, thereby improving the model’s discriminative power and generalization ability. Moreover, we have constructed a KDC fusion block based on attention mechanism that extracts features both spatially and channel-wise. Channel attention is derived from the original image, whereas spatial attention is extracted from multi-source pathological nuclei regions. This differs from previous attention modules, which typically use only the same image to extract both channel and spatial attention. Through this method, we can extract tumor-specific features from different perspectives, achieving a mutually complementary and enhancing effect.

A new idea to integrate deep learning features and handcrafted features for the combined use of heterogeneous features from multiple sources. MSFusion integrates the strengths of deep learning features with handcrafted features. The former autonomously captures complex and abstract representations within data, while the latter incorporates specific morphological information based on prior knowledge. These two types of features complement each other, thereby enhancing the model’s representational capability, improving data utility efficiency, and ultimately optimizing the overall performance of the model. In terms of handcrafted feature extraction, considering the varying nuclear spatial arrangements across different pathological regions, this study specifically extracted features for tumor nuclei, mitotic nuclei, tubular nuclei, and epithelial nuclei. For each specific pathological region, the team extracted handcrafted nuclear features across six dimensions: Cell Architecture, Cell Morphology, Cell Orientation Entropy (COrE), Cell Cluster Graph (CCG), Haralick, and Quantitative Relationships (QR). This method differs from previous approaches that focused solely on a single region or extracted handcrafted features from just one dimension, allowing for a more comprehensive description of the heterogeneity and complexity of nuclei.

To validate the effectiveness of the MSFusion method, we conducted extensive experiments on two private clinical datasets and one public dataset, comparing them with state-of-the-art methods. These experimental results not only showcased the superior performance of our method but also confirmed its applicability for processing actual clinical images.