We previously reported an acceptable specificity of the four automated Ag tests for SARS-CoV-2 evaluated in the current study, ranging between 97.0 and 99.7% [32]. Here, we tested the analytical sensitivity of these automated Ag test systems to detect different dominant Omicron subvariants, specifically BA.1, BA.2, BQ.1 and XBB.1.5 as well as other minor subvariants. We included 203 SARS-CoV-2 PCR-positive nasopharyngeal swabs, of which 53 were classified as BA.1, 48 as BA.2, 23 as BQ.1, 39 as XBB.1.5 and 40 as other minor subvariants, respectively, and compared those to the 107 “non-Omicron” SARS-CoV-2 swabs, results for which were reported recently [32]. In contrast to the initial “non-Omicron” data set with a viral load range of 83 to 1,548,572,803 Geq/ml (median: 6,045 Geq/ml), the swabs taken from patients with these Omicron subvariants showed considerably higher viral loads ranging from 3,361 to 1,824,858,018 Geq/ml for the Omicron subvariants from 2022 (median: 373,560 Geq/ml) and 3,129 to 832,225,970 Geq/ml for the Omicron subvariants from 2023 (median: 762,693 Geq/ml) (Fig. 1).

Fig. 1

SARS-CoV-2 viral load distribution of respiratory samples included in the study. A Shown is the log10 viral load (Geq/ml) of 203 SARS-CoV-2-positive (101 Omicron 2022; red and 102 Omicron 2023; blue) versus 107 SARS-CoV-2-positive (“non-Omicron”; black [32]) patient samples, sorted by ascending magnitude from left to right. Each dot indicates one patient and the sample’s ID is indicated. B and C Depicted is the histogram of the viral load distribution by categorization of samples into defined log10 viral load value ranges. Each bar indicates the number of samples in the respective viral load range; B for Omicron 2022 samples (red), C for Omicron 2023 samples (blue). D The horizontal line in the box plots shows the median of the samples shown in A, bound between upper and lower quartiles, and whiskers between minimum and maximum are indicated. *p < 0.05, ****p < 0.000005 by Wilcoxon rank sum test with continuity correction and by two-sample Kolmogorov–Smirnov test

Diagnostic sensitivity of automated antigen tests for Omicron subvariants

We then compared the diagnostic sensitivity (Table 2) using the Ct/Cp value ranges of respiratory swabs for stratification, and analytical performance (Fig. 2) of the four automated SARS-CoV-2 Ag tests. To determine the analytical performance, we used the cutoffs for “positive” and “negative” scoring according to the manufacturers’ recommendations (CLEIA ≥ 1.34; CLIA ≥ 200; ELISA ≥ 0.60; ECLIA ≥ 1.00; grey dashed lines).

Table 2 Comparative evaluation of the diagnostic sensitivity of four automated SARS-CoV-2 Ag tests stratified for Ct/Cp value ranges for “non-Omicron”* and Omicron-containing respiratory samplesFig. 2

Analytical sensitivity of PCR-positive SARS-CoV-2 patient samples for quantitative SARS-CoV-2 Ag tests. A CLEIA from Fujirebio, B CLIA from Diasorin, C ELISA from Euroimmun and D ECLIA from Roche Diagnostics. Omicron 2022 samples are shown in red, Omicron 2023 samples in blue and data for “non-Omicron” samples retrieved from our previous study [32] are shown in black. The log10 of quantified samples were plotted against the log10 of the calculated viral loads. The grey dashed line indicates the cutoffs recommended by the manufacturers

When stratifying the binary (“positive”, “negative”) test results according to the samples’ Ct/Cp value ranges, all four automated SARS-CoV-2 Ag tests were able to score true-positive results for at least 91.7% of Omicron-containing samples in 2022 with high viral loads (Ct < 25) (Table 2). The rates of true-positive results with intermediate viral loads (Ct 25–30) varied between 6.7% and 100.0%, while they decreased to 0 to 15.4% for samples with low Ct values (Ct > 30). For the Omicron subvariants in 2023, at least 90.9% of the samples were scored positive for high viral loads (Ct < 25), while those values varied markedly between 0 and 100% for both intermediate and low viral loads. Here, detection of BQ.1 by ELISA was worst among these categories. In contrast, the diagnostic detection of the “non-Omicron” samples was more sensitive for CLEIA, CLIA and ELISA, scoring positive in 35 to 100% (Ct 25–30) and 1.3 to 32.9% (Ct > 30) of cases, respectively (Table 2). Interestingly, Omicron-positive samples collected in 2023 for CLEIA and ECLIA were detected more readily compared to the Omicron samples collected in 2022 as well as the anecdotal cohort. It is of note that diagnostic sensitivities for BA.1, BA.2 and BQ.1 detection were comparable, although a trend towards a reduction in performance for the latter VoC was noted, while samples containing XBB.1.5 were superior in this respect. For the subsequent experiments and analyses, data for the Omicron subvariants from specimen sampled in 2022 and 2023 were combined, respectively, and compared to the “non-Omicron” samples. Most notably though, the diagnostic sensitivities of the four automated Ag test systems differed substantially among each other, in agreement with our previous report [32].

Receiver operator characteristic analysis and evaluation of different cutoff values for Omicron sublineages

We next evaluated receiver operator characteristic (ROC) curves of the four automated Ag tests, comparing their performance between Omicron 2022/2023- and “non-Omicron”-containing respiratory samples. CLEIA was best for the “non-Omicron” specimens with a calculated area under the curve (AUC) of 0.873 [32]. Using the Omicron 2022 (red) and 2023 (blue) samples for ROC evaluation, the AUC for CLEIA were even better, i.e., 0.986 (Fig. 3A). In comparison, the AUC for CLIA remained rather low with 0.565 for the 2022 samples while improving to 0.891 for the 2023 samples; for comparison, the “non-Omicron” AUC was 0.516 [32] (Fig. 3B). Interestingly, ELISA and ECLIA also showed high AUCs with 0.804 and 0.810 for the 2022 samples and 0.993 and 0.990 for the 2023 samples, respectively (“non-Omicron” ELISA: 0.650, “non-Omicron” ECLIA: 0.670 [32]) (Fig. 3C, D).

Fig. 3

ROC analyses for quantitative SARS-CoV-2 Ag tests with A CLEIA, B CLIA, C ELISA and D ECLIA. The respective AUCs are depicted. Omicron 2022 samples are shown in red, Omicron 2023 samples in blue and data for “non-Omicron” samples retrieved from our previous study [32] are shown in black

Based on the ROC analyses, we re-evaluated the cutoff values with our Omicron data set. We calculated the specificities and sensitivities, setting the WHO’s minimal criteria of > 80% sensitivity and > 97% specificity, respectively (Tables 3, 4). To reach a specificity of 97%, the sensitivity of CLEIA had to be lowered to 82.2% for the 2022 samples, while it was 91.0% for the 2023 samples (Table 3). Thus, CLEIA was always able to fulfill the WHO ‘s minimal criterion for sensitivity > 80% under conditions of acceptable specificity (Table 4). For the Omicron samples in 2022, CLIA’s sensitivity would stay rather unaffected compared to the optimal cutoff setting within the non-Omicron cohort upon adjustment of the specificity to 97% (41.6%, Table 3). On the contrary, specificity would be only 10.9% when increasing sensitivity to 80% (Table 4). For the Omicron samples in 2023, sensitivity improved to 68.6% for a specificity of 97%. To reach a sensitivity of 80%, the specificity for the Omicron samples in 2023 had to be markedly lowered to 71.3%. For ELISA, an increase in specificity to 97% with the Omicron samples in 2022 would thus not have a major impact on the sensitivity compared to the optimal non-Omicron-cutoff setting (52.6%, Table 3). Aiming for a sensitivity of 80% would, however, lower the specificity to 49.4% (Table 4). The Omicron samples from 2023 were superior in that respect: aiming for a specificity of 97%, the sensitivity was 94.3% and for a sensitivity of 80%, the specificity was 99.7%, respectively, fulfilling thereby the WHO’s minimal criteria. Compared to the optimal non-Omicron-cutoff setting, the sensitivity of ECLIA for the Omicron samples in 2022 would only be marginally reduced to 55.0% if a specificity of 97% should be achieved (Table 3). Aiming for a sensitivity of 80% would decrease the specificity to an unacceptable level of 49.5% (Table 4). Here, again, the Omicron samples in 2023 fulfilled the WHO’s minimal criteria: To reach a specificity of 97%, the optimal sensitivity was 91.1%, while the optimal specificity was 99.3% when setting the sensitivity to 80%.

Table 3 Cut-off-dependent calculation of extrapolated sensitivities using the WHO minimal criteria with specificity fixed at 97.0%Table 4 Cut-off-dependent calculation of extrapolated specificities using the WHO minimal criteria with sensitivity fixed at 80.0%Analytical sensitivity of automated antigen tests for Omicron sublineages

Next, we calculated the 50% and 95% limits of detection (LoDs) based on a logistic regression model as reported recently [59]. The virus concentrations at which 50% (LoD50) and 95% (LoD95) detection rates were obtained for CLEIA with the Omicron samples in 2022 corresponded to 44,444 Geq/ml and 154,621 Geq/ml, while the ones for the Omicron samples in 2023 were 29,951 and 227,484 Geq/ml, respectively (“non-Omicron”: LoD50—6,181 Geq/ml, LoD95—422,689 Geq/ml [32]; Fig. 4A). The LoD50 and LoD95 of CLIA were 42-fold higher compared to CLEIA, respectively, and equaled 1,866,900 and 6,433,679 Geq/ml for the Omicron samples in 2022 (“non-Omicron”: LoD50—473,279 Geq/ml, LoD95—11,452,782 Geq/ml [32]; Fig. 4B). For the Omicron samples in 2023, the differences for LoD50 and LoD95 were comparable with 38-fold and 87-fold, respectively, which corresponded to 1,152,048 and 19,824,213 Geq/ml. ELISA yielded LoD50 and LoD95 with 1,069,098 and 1,742,957 Geq/ml for the Omicron samples in 2022, which were 24-fold and 11-fold higher compared to CLEIA, respectively (“non-Omicron”: LoD50—749,792 Geq/ml, LoD95—25,711,669 Geq/ml [32]; Fig. 4C). Those for the Omicron samples in 2023 were in a similar range with 27-fold and 61-fold higher values compared to CLEIA, respectively, corresponding to 794,733 and 13,895,613 Geq/ml. ECLIA was only sixfold and ninefold inferior to CLEIA for the Omicron samples in 2022 and resulted in 281,582 and 1,380,738 Geq/ml for LoD50 and LoD95, respectively (“non-Omicron”: LoD50—69,002 Geq/ml, LoD95—2,654,696 Geq/ml [32]; Fig. 4D). For the Omicron samples in 2023, LoD50 and LoD95 were comparable to CLEIA with 23,019 (LoD50) and 276,728 (LoD95) Geq/ml. It is of note that no statistically significant difference between the three cohorts, “non-Omicron”, Omicron-2022 as well as Omicron 2023, could be detected for all four automated Ag tests.

Fig. 4

Limit of detection analyses of PCR-positive SARS-CoV-2 patient samples for quantitative SARS-CoV-2 Ag tests: A CLEIA, B CLIA, C ELISA, and D ECLIA. Omicron 2022 data set is shown in red, Omicron 2023 in blue and data for “non-Omicron” samples retrieved from [32] are shown in black. The log10 viral load of quantified samples on the x-axis was plotted against a positive (+ 1) or negative (0) test outcome on the y-axis. For readability of the figure, slight normal jitter was added to the y-values. Red/grey/blue curves show logistic regressions of the viral load on the test outcome; vertical dashed lines indicate log viral loads at which 50% (LoD50) and 95% (LoD95), respectively, of the samples are expected positive based on the regression results

Focusing on the individual Omicron sublineages from 2022 and 2023, the LoD analyses revealed that CLEIA was up to sevenfold (LoD50) and up to twofold (LoD95) inferior regarding the detection of BQ.1-positive samples compared to the other subvariants. In contrast, ELISA was up to sixfold (LoD50) and ECLIA up to 36-fold (LoD50) superior in scoring XBB.1.5-positive samples positive, respectively (Fig. 5; Table 5). However, it must be stated that for these lineage subgroups the sample numbers were very small limiting the conclusions to be drawn.

Fig. 5

Limit of detection analyses of PCR-positive SARS-CoV-2 patient samples for quantitative SARS-CoV-2 Ag tests: A CLEIA, B CLIA, C ELISA, and D ECLIA. Omicron 2022 data set (subdivided in BA.1 and BA.2) is shown in red and Omicron 2023 (subtypes BQ.1 and XBB.1.5) samples are shown in blue. The log10 viral load of quantified samples on the x-axis was plotted against a positive (+ 1) or negative (0) test outcome on the y-axis. For readability of the figure, slight normal jitter was added to the y-values. Red/blue curves show logistic regressions of the viral load on the test outcome; vertical dashed lines indicate log viral loads at which 50% (LoD50) and 95% (LoD95), respectively, of the samples are expected positive based on the regression results

Table 5 Summary of non-synonymous amino acid substitutions with a prevalence of > 75% in the nucleocapsid protein of the SARS-CoV-2 VoCs examined in this study in publicly available VoC sequences compared with the original Wuhan-hu-1 sequence.Reevaluation of the measurement kinetics of the automated antigen tests for PCR-negative samples

It is striking that for the Omicron samples from 2023 the ECLIA achieved a significantly higher sensitivity than in the previous cohorts. We, therefore, compared the measurement behavior of the ECLIA with the ELISA and CLIA, as they showed similar test kinetics (Fig. 2). The CLEIA had an extremely broad dynamic measurement range that continued much further below the cutoff than the CLIA, ELISA, and ECLIA. Therefore, CLEIA was excluded from the following analysis. To obtain further information on characteristics of ECLIA, the measurement results of the 303 PCR-negative samples from the non-Omicron cohort were re-analyzed [32]. Using the manufacturer's recommended cutoffs, the mean relative deviation from the cutoff for the ECLIA measured values was -22.59%, which is only 1.91 times the standard deviation. For CLIA and ELISA, the mean deviations from the cutoff were significantly higher at -53.98% (8.25-fold standard deviation) and −56.88% (4.28-fold standard deviation), respectively.

Comparable detection of tissue culture-expanded VoCs by quantitative, automated SARS-CoV-2 antigen tests

In our final approach to characterize the performance of the four automated Ag tests, we extended our analyses to tissue culture-expanded Delta, Omicron-BA.1, -BF.7, -BN.1, and -BQ.1 clinical isolates (Fig. 6A-D). While all of these expanded VoCs could be detected by these four tests, their performance differed widely: for CLEIA, the investigated VoCs scored positive with concentrations ranging between 135,000 Geq/ml (wt and BN.1) to 2,170,000 Geq/ml (Delta and BF.7). Both for CLIA and ELISA, higher concentrations between 2,170,000 Geq/ml (BF.7) and 34,645,828 Geq/ml (Delta) were necessary, similar to observations made for patient swab specimens. When analyzing the ECLIA, we noticed that the values at low concentrations fluctuated around the manufacturer’s cutoff, especially true for wt, Alpha, Beta and BN.1. Of note, in diluted and inactivated mock cell culture samples, positive signals around the cutoff were also measured, similar to the measurements of the VoCs (data not shown). This indicates that the ECLIA records non-specific, low positive signals with this type of culture medium. For the ECLIA, samples were scored positive with concentrations ranging from 135,000 Geq/ml (BQ.1, BN1, BF.7) to 8,660,000 Geq/ml (Delta).

Fig. 6

Evaluation of VoCs isolated in cell culture experiments using the (semi-)quantitative SARS-CoV-2 Ag tests: A CLEIA, B CLIA, C ELISA and D ECLIA. The measured nucleocapsid protein results were plotted against the calculated SARS-CoV-2 RNA loads. The grey dashed line indicates the cutoffs recommended by the manufacturers