The evaluation of retrospective data was approved in accordance with the ethical standards of the Chinese PLA General Hospital Committee.

From January 2016 to April 2018, we retrospectively reviewed data of 113 patients with inconclusive PNs on [18F]FDG PET/CT, all of whom underwent additional preoperative [18F]FLT PET/CT scans within a week [22]. An inconclusive PN was considered when the following criteria were met: (1) the lesions have apparently higher SUV than the rest of the lung; (2) there is no obvious evidence of nodal or distant metastasis, and (3) there are no definite indications of morphological malignancy, such as air bronchograms, spiculated or irregular margins, or lobulated shape.

The inclusion criteria of the study were (a) radiologically clear propensity to be diagnosed as a pulmonary nodule with a diameter of no more than 3 cm, (b) no definite diagnosis prior to [18F]FDG and [18F]FLT PET/CT examinations, (c) no treatments before PET/CT examination, (d) no indications of major organ dysfunctions or disorders, and (e) clear histopathologic identification or the endpoint of long-term follow-up.

[18F]FDG and [18F]FLT were produced, and both their radiochemical purities are higher than 95%. Every patient fasted for over 4 h with a blood glucose level of < 11.1 mmol/L and rested in a quiet room for about half an hour. Then, the [18F]FDG tracer was given intravenously in a standardized dose of 3.70–4.44 MBq/kg. An hour after the tracer administration, every patient underwent [18F]FDG PET/CT scan (Discovery ST; GE Healthcare), and at least 1 day after [18F]FDG PET scan, the [18F]FLT tracer was also injected at a dose of 3.70–4.44 MBq/kg, and an hour later, every patient underwent [18F]FLT PET/CT scan (Discovery ST; GE Healthcare).

For both tracers’ scans, we ran the following settings to get low-dose CT (LDCT) scans to prevent patients from excessive radiation: 120 kV, 100–250 mAs with automatic adjustment, 0.8 s rotation, 1.25 mm collimation, and a pitch varied according to the geometry of CT detector (4, 8, or 16 slices). Meanwhile, PET was scanned in 2 min/bed, 3- or 4-bed positions (axial field view 15.7 cm) in three-dimensional mode with three iterations, and 21 subsets. Then, images were acquired. All the PET/CT data were reconstructed with the Fourier rebinding iterative algorithm and a Gaussian filter of 4 mm full width at half maximum.

Three clinicians with more than 10 years of diagnostic experience in pulmonary diseases conducted the visual analysis of PET/CT images. All the PET/CT image interpreters were blind to the patients’ information. The interpretation of PN malignancy was listed in Additional file 1. Also, the final diagnosis was determined based on all listed characteristics. Discrepancies between interpreters were resolved in a consensus meeting.

All of the PET/CT data were analyzed by a semi-automated adaptive threshold method at the RadCloud platform (Huiying Medical Technology Co., Ltd., Beijing, China). Volumes of interest (VOIs) of the PN were initially drawn with a threshold of 40% of the SUVmax according to PET images via a commercial software (PET VCAR, GE Healthcare, Waukesha, WI, USA). After that, VOIs were checked visually on whether they have covered the whole components shown on the CT. If not, a lower threshold was then used [23]. If the VOI contains surrounding physical tissues, such as the adjacent myocardial activity, we would adjust its boundary manually [24]. All the segmentation was conducted by the same handler (a nuclear medicine physician with 4 years of experience in tumor drawing).

Both of the CT and PET images were analyzed, where radiomic feature calculations were performed within the same VOI in the settings of MATLAB (The MathWorks Inc.). When the respiratory motion leads to mismatches between CT and PET images, the extension of the VOI would be manually adapted to CT images. Before the computation of radiomic features, image voxel intensities were resampled into equally spaced bin widths of 0.1 [25]. The radiomic features extracted from [18F]FDG PET/CT images include shape features, histogram-based features, and texture features. Furthermore, texture features covered gray level cooccurrence matrix (GLCM), run length matrix (GLRLM), size zone matrix (GLSZM), and neighborhood gray-tone difference matrix (NGTDM) features. For more information about the description of the texture feature, see Additional file 1.

To build the radiomic models, we took the histologically confirmed malignant or benign PN as the ground. In detail, firstly, the dataset was randomly divided into a training set and a test set in a ratio of 7:3, of which the training set was used for feature selection and modeling while the test set was used to test the models’ performance (the distribution of the dataset is shown in Table 1). Secondly, an analysis of variance (ANOVA) was applied to univariate feature selection to clarify the value of image features in the dataset in the differentiation of benign PNs from malignant. To be more detailed, features were eliminated if the p value exceeded 0.05. Then, the least absolute shrinkage and selection operator (LASSO) method was applied to the high-dimensional data regression to screen out the most valuable discriminative features from the training set [26], to prevent machine learning models from overfitting. The minimum mean square error (MSE) was calculated through fivefold cross-validation. Based on MSE under different parameters, the best penalty parameters and fitting model of LASSO were obtained. Finally, the non-zero coefficient features were selected for model training.

Two radiomics-based models were established using logistic regression (LR) for the distinguished diagnosis of benign PNs from malignant. The radiomic features selected by ANOVA and LASSO on [18F]FDG PET/CT images were used for modeling. Meanwhile, we classified PNs’ uptakes indicated on [18F]FLT PET images into no uptake, slight uptakes (lower than half the value of thoracic vertebrae), and apparent uptakes. In the model of FDG-based radiomic analysis, the threshold of the predictive probability value is 0.699. In the model of radiomic analysis based on FDG and FLT, the threshold of predictive probability value is 0.594. That is, if the value is more than the threshold, then the PN was regarded as malignant and otherwise it was benign. The classification was regarded as a signature, which was then integrated with the selected features of [18F]FDG PET/CT images to build another model. By doing so, we tried to find out whether the modeling performances could be improved. At last, the performances of the two models were tested with the 5-fold cross-validation method and then quantitatively assessed through the area under the curve (AUC), sensitivity, and specificity based on the receiver operating characteristic (ROC). Besides, a separate test set was run for verification.