A review of 119 research articles on AI algorithms for diagnosing lung cancer and lung diseases using medical images was conducted. Out of these, 108 articles introduced novel AI algorithms, referred to as "New Algorithms". Most of these algorithms used a combination of four foundational models: CNN, GAN, other NN (other derivatives of NN), and ML (conventional non-NN architecture). Furthermore, fuzzy ML and metaheuristic search optimization algorithms were integrated as complementary techniques to enhance the performance of the new algorithms.

A consistent pattern prevailed throughout the observed articles: a strong emphasis on utilizing CNN-based models or components. Many researchers either devised entirely new CNN algorithms tailored for specific diagnostic purposes or enhanced existing publicly available CNN models by incorporating customized or modified layers. However, beyond this predominant pattern, several noteworthy trends were identified over the years:

i.

Direct Application of Transfer Learning: A prevalent strategy involved directly adopting readily available CNN models, such as VGG, ResNet, and AlexNet and leveraging transfer learning techniques to fine-tune them for lung disease and cancer diagnosis was observed [31,32,33,34,35].

ii.

Integration of GANs: Some researchers [36,37,38,39,40] incorporated GANs into their approaches for image synthesis purposes to enlarge the size of training data for better model generalization, some authors [11, 41,42,43] applied GANs for segmentation purposes, and some authors [44,45,46,47] applied GANs for classification purposes.

iii.

Diversification with Different Neural Network Models: In addition to CNNs, other NN-based models were explored [48,49,50]. It was noted that in most cases, these studies incorporated other NN-based models in conjunction with CNNs rather than employing them as standalone models.

iv.

Exploration of Non-Neural Network Models: A distinct approach was taken by some researchers who ventured beyond NN, experimenting with conventional ML approaches or alternative models to devise effective diagnostic solutions [51,52,53,54]. Similar trends were noted in the context of other NN models, where the prevailing approach involved employing traditional ML classifiers to classify extracted features in the final phase of the CAD system after the image processing phase, in which a foundational CNN model was utilized.

v.

Hybrid Model Development: A trend emerged where hybrid models were built, combining different AI and/or non-AI techniques to harness their collective strengths for improved diagnostic outcomes [11, 30, 55,56,57,58,59].

vi.

Utilization of Commercial CAD Tools: Additionally, some studies [60,61,62,63] examined the commercially available CAD tools to accurately diagnose lung diseases.

4.1 Categorization of the New Algorithm into distinct model groups

All the New Algorithms have been systematically classified into the 7 distinct model groups, as outlined below:

(i)

Pure CNN: Entries constructed solely using CNN architecture, without integrating optimization algorithms and/or Fuzzy ML.

(ii)

Pure GAN: Entries constructed solely using GAN, without integrating optimization algorithms and/or Fuzzy ML.

(iii)

Pure Other NN: Entries constructed solely using Other-NN, without integrating optimization algorithms and/or Fuzzy ML.

(iv)

Pure ML: Entries constructed solely using ML, without integrating optimization algorithms and/or Fuzzy ML.

(v)

Other: Entries constructed solely using approaches other than those among CNN, GAN, Other-NN, and ML, without integrating optimization algorithms and/or Fuzzy ML.

(vi)

Commercial CAD system: Entries that applied commercially available CAD systems or prototypes.

(vii)

Hybrid ML: Entries meeting at least one of the following conditions:

a.

Combining two or more foundational models from .

b.

Combining one foundational model from with other ML methods.

c.

Incorporating one foundational model from along with an optimization algorithm and/or Fuzzy ML.

This meticulous categorization scheme effectively encompasses all 108 New Algorithms, providing a clear framework for their analysis and comparison throughout the subsequent sections. Figure 6 provides insight into the distribution of the New Algorithms across the 7 model groups, illustrating the proportional representation within each. Further, Fig. 7 offers a chronological view, arranging the 108 New Algorithm entries into the 7 model groups according to their publication years, whereas Fig. 8 displays according to their corresponding model groups.

Fig. 6

Distribution of new algorithms across 7 model groups

Fig. 7

Distribution of model groups of the new algorithm by year of publication

Fig. 8

Categories of new algorithm entries by model group types

Figures 6 and 7 reveal a noteworthy pattern, in which both Pure CNN and Hybrid ML garner significant attention, comprising the largest segments in the pie chart. An intriguing observation is the pronounced focus on Pure CNN since 2019, coinciding with the rise of Hybrid ML studies. Notably, Hybrid ML experienced a marked surge in 2023. This surge can be attributed to the recognition that Pure CNN may not comprehensively address the demands of CAD workflows and often lacks optimal generalization within end-to-end CNN frameworks. To address these limitations, numerous investigations [9, 15, 58, 59, 64,65,66,67,68,69,70,71,72,73,74] adopt multi-stage models, merging diverse approaches into a single framework. Number of studies [11, 14, 46, 55, 57, 75,76,77,78,79,80,81,82,83,84,85,86,87] navigate this challenge by leveraging metaheuristic search techniques for hyperparameter optimization, mitigating the prolonged training issue. The observed trend suggests a continued influx of novel Pure CNN and Hybrid ML algorithms, driven by the ongoing pursuit of refined and specialized CAD solutions.

4.2 An Even More Comprehensive Categorization of the New Algorithms Based on Foundational Models and Methodology

This section provides a breakdown of how the New Algorithms entries are distributed by their constituent models and methodologies. Figure 9 presents a diagram where the size of the regions reflects the number of entries falling within each specific combination. The pink shading represents the entries categorized under Hybrid ML, while the black numbers correspond to entries falling within model groups numbered 1 to 6, as defined in Section A.

Fig. 9

Euler diagram illustrating the distribution of new algorithm entries based on foundational models and methodologies

Optimization and fuzzy instances are not considered distinct foundational models because they are typically used as accompanying algorithms rather than standalone entities within New Algorithms. Thus, foundational models that were attached with optimization algorithm and/or Fuzzy ML are classified as Hybrid ML.

Several studies involve integrating metaheuristic optimization algorithms to enhance network performance. These investigations are represented in the section bounded by the box in green border in Fig. 9, and these studies are detailed in Table 2. Furthermore, the exploration of Fuzzy ML within this domain is limited, with only two entries [30, 56] incorporating Fuzzy ML alongside primary models. In contrast, both Hybrid ML and Pure CNN investigations have gained significant attention, each with 42 entries.

Table 2 Classification of all the 108 new algorithms based on foundational models

Pure CNN has advantages in extracting informative deep features and operating seamlessly as an end-to-end model, making it more user-friendly for clinical applications. However, practical applications in medical image processing are often complex due to the high noise levels in raw images. This requires preprocessing, segmentation, and detection steps before they can be used for optimal classification and diagnostic outcomes. Hence, a significant portion of research work remains focused on improving the established CAD workflow. This explains the prevalent interest surrounding the exploration of Hybrid ML and Pure CNN.

4.3 In-Depth Review of the Model Category: Pure CNN

This section analyzes 43 Pure CNN entries, exclusively constructed upon CNN models. The objective is to examine each entries’ unique characteristics, design principles, and applications, offering readers a profound comprehension of CNN-driven algorithms.

4.3.1 CNN Architectures Overview

CNNs are the preferred method for computer vision, especially in CAD for lung cancer, due to their specialized design for grid-like data [134]. Figure 10 shows the core foundational elements collectively adopted by Pure CNN models.

Fig. 10

Common structures of a typical CNN framework

In the CNN workflow, the process involves two main parts: extracting features from input images and classifying them. Different layers such as convolutional, pooling, and fully connected layers perform various operations on the input data.

Convolutional layer. This layer extracts features from 3D tensor data through convolution operation to produce feature maps.

Pooling layer. This layer adeptly reduces the dimensionality of the feature maps, facilitating a more compact representation.

Fully connected layer. This layer forms connections between each neuron in the preceding layer and the current layer, producing a feature vector representing the final model prediction.

4.3.2 Utilization of Transfer Learning

CNN models require significant computational resources and extensive training data to achieve optimal performance. Transfer learning allows a model to leverage existing knowledge and customize it for a specific domain, such as classifying lung cancer images. In basic terms, the model gains knowledge through saved weights and then is further trained in a specialized field to excel in that domain. This speeds up learning as opposed to starting from scratch.

Publicly available CNN models, such as VGG, ResNet, DenseNet, MobileNet, AlexNet, and Inception, have been extensively trained on natural images from the ImageNet dataset and can be useful for transfer learning. Several investigations [31, 32, 34, 35, 59, 92, 96, 100, 114,