This study has several important findings. First, we demonstrate that SWPs vary widely in low end accuracy based on measurement of rMBF in regions of known TMS. Second, we confirm the “low flow” accuracy and test–retest precision of a modern 3D PET-CT, via examination of a reference region of dense TMS using the software package HeartSee. Third, we describe a method of “converting” a ROI of TMS to segmented ROI and provide reference values whereby myocardial blood flow within transmural infarctions can be easily measured even within the significant limitations of the standard 17-segment model. Fourth, we report that while HS and, for a large extent, Emory-O, yielded values consistent with transmural scar, other SWPs consistently overestimated rMBF within dense scar. Our data also provide insights into the mechanism(s) for these SWPs’ upward biases. Finally, we establish that the same-day test–retest variability of rMBF in patients with TMS is ~ 7–9% using a modern 3D PET-CT.

There are numerous publications comparing various quantitative SWP. However, when between-SWP discordances are present, the user is left uncertain which (if any) is accurate [13, 23,24,25,26]. Current recommendations that suggest each laboratory set their own thresholds for quantification [27, 28] are antithetical to the very nature of quantitative PET [29]. The question remains, “What simple means exist for PET laboratories to determine accuracy of SWP?”.

When evaluating the performance of quantitative PET imaging, a variety of test cases are available to clinicians, with various flow characteristics: (1) “normal” MBF in healthy volunteers (2) “ischemic” sMBF and (3) rMBF in transmural scar. “Normal” MBF flow is too heterogenous and with standard deviations too wide to be used as a reference, whereas “ischemic” flow values also will vary depending upon the definition of “ischemia” [30, 31]. However, resting flow in transmural scar is not physiologically burdened with this wide variability, and has several other favorable characteristics. First, by definition, the myocardium within the transmural scar is dead, which can be confirmed with adjunct advanced testing such as MRI or FDG or based on history and common clinical studies (i.e., echocardiography, ECG, SPECT). Second, TMS is easy to identify, as there is a severe defect within the region of analysis. Gupta et al., for example, have used a threshold of ≤ 50% maximum uptake as a definition of transmural scar [32]. As noted, Table 2 demonstrates that the mean uptake of infarcts in the current study was 41–47% for all SWPs. Third, rMBF within the scar (by definition) should be lower than living viable tissue. Various studies authored by Beanlands et al., Benz et al., Wang et al., and Zhang et al. determined viability rMBF “thresholds” of 0.45, 0.45, 0.42, and 0.42 mL/min/g, respectively [2, 10, 12, 33]. Below these thresholds, non-viable, dead myocardial scar must be present.

However, these noted studies did not discriminate scar thickness (i.e., between transmural and non-transmural scar). In one of the most compelling studies on this topic, Rivas et al. [3] performed a transmural analysis with microspheres from endocardial through epicardial layers in infarcted dogs. They found a gradient of rMBF ranging from 0.00 to 0.35 mL/min/g in layers of myocardium with scar thickness > 72%. Furthermore, as scar thickness decreased (non-transmural scar became predominant), rMBF increased (rMBF ranged from 0.36 to 0.75 mL/min/g in non-TMS). The greater the scar thickness, the lower the rMBF [3]. Similar to Rivas, Stewart et al. also demonstrated an inverse relationship between scar thickness and rMBF. They found that mean rMBF within non-TMS was 0.45 ± 0.14 mL/min/g and within TMS was 0.32 ± 0.07 mL/min/g. In this study, cardiac MRI was used to determine scar thickness and PET was used to determine rMBF in a ROI of scar. Thus, as myocardial scar thickness increases, not only does the mean rMBF decrease but also does its standard deviation. Grönman et al. [34] made similar observations with [O-15]H2O. However they did not specifically distinguish between TMS and non-TMS, but did find a mean rMBF of 0.45 mL/min/g. Examples within their report demonstrate rMBF < 0.30 mL/min/g within the most severe segments of scar [34].

Based on the constellation of data, reference values for rMBF within regions of TMS should not be in dispute as they were validated by invasive, histologic, and non-invasive means and summarized in Table 5. These reference values has been verified not only by PET using various radiotracers and kinetic models (15O-water, 13N ammonia, and 82Rb) in both human and animal studies, but also using the gold standard: microspheres [1,2,3,4,5,6,7,8,9,10,11,12]. The results of these studies utilizing multiple techniques, models, and radiotracers all consistently converge on a narrow range of 0.30 mL/min/g or less, with an upper limit of 0.44 mL/min/g (when using a segmental analysis), thus suggesting a universal limit, reference range, or “gold standard.” As a firmly established constant, all combinations of techniques, correctly functioning scanners (2D or 3D), kinetic models, and radiotracers should describe TMS rMBF within this narrow reference range. In contrast, measurements outside of this established reference range (i.e., rMBF of 0.5 mL/min/g or higher) would suggest an upward bias of the SWP, inaccuracies of the imaging system, or perhaps imaging non-TMS.

Thus, rMBF within dense TMS is an optimal reference for determining software bias, because its true value is within a known narrow margin and regions of TMS are easily identified in clinical practice.

It is worth reiterating that we selectively imaged regions of myocardium with dense transmural scar, where all questions regarding any possibility of viability had been previously settled. We were not testing any hypotheses for a possible rMBF threshold for viability, nor did we focus on any region with non-transmural scar. All scans demonstrated a near absence of radiotracer in the region of TMS, and each participant’s clinical history was consistent with a large, dense, transmural scar without evidence of viability. Thus, every patient who underwent PET/CT scanning for this study had established transmural scar without any evidence of viability.

Performance differences between SWP have been a prominent topic in the literature, but “the absence of a gold standard by which to judge…accuracy” has been a major limitation [36]. Kamphuis et al. have proposed an advanced phantom pump to establish a “ground truth validation of absolute MPI applications in the clinical setting” [37]. While Bui et al. [17] have used such a phantom pump in establishing a “ground truth” for the arterial input function, their method does not test implementation of other aspects of kinetic models such motion, spillover, and boundary segmentation. Furthermore, such phantom pumps are not readily available, are expensive, and require a level of expertise that is beyond the capacity of most PET labs. In this study, we employed a simple inexpensive method for assessing PET systems and SWP low-end accuracy in clinical settings- namely a dense region of transmural myocardial scar without evidence of viability. Furthermore, most cardiac PET laboratories can utilize the methods described here, without additional equipment, complex pumps, or involvement in research protocols.

Using HeartSee on a contemporary 3D PET-CT, we report that the median rMBF in a ROI of TMS is 0.26 mL/min/g, with median minimal rMBF of 0.17. These expected values suggest that both camera and software are functioning appropriately.

As noted, HS provides a ROI tool such that the exact contours of the infarct can be selected by software algorithm. However, the remaining tested SWPs did not have such a feature. Thus, the only method for detailed defect analysis in non-HS SWPs was via the standard 17-segment model. The 17-segment model is suboptimal for the required analysis as most scars affect only parts of individual segments, thus biasing readings toward higher rMBF values. Therefore, to uniformly compare all SWPs, we determined a segmental equivalent (Seg-Scar) to the ROI of TMS (ROI-Scar) and provided reference values whereby transmural infarctions can be measured within the limitations of the non-physiologic segmentation employed by the standard 17-segment model. Based on our methods, median rMBF (mL/min/g) in a “segmented” TMS is 0.29 [0.26–0.40], with the lowest segment measuring 0.22 [0.19–0.28] and with an expected ~ 15% of segments showing rMBF > 0.44 (Figs. 4b and 6). Until other SWPs implement an ROI tool, our segmental methodology may be used, as it is easy to employ and functional.

In contrast to HeartSee and Emory-O, the other SWPs demonstrated consistent upward bias of rMBF for scar-related rMBF. Among the 60 scans in this study, 4DM, 4DM-FDV, Cedars-Sinai and Emory-V returned rMBF values consistent with TMS (i.e., < 0.44 mL/min/g) in 15%, 62%, 20% and 30% of scans, respectively.) Median rMBF (mL/min/g) in Emory-O Seg-Scar was significantly higher than HeartSee Seg-Scar (0.37 vs. 0.29, p = 0.006); however, there was only a 5% difference between the two SWP in values inconsistent with TMS. One could hypothesize that if Emory-O had an ROI tool, ~ 5% of cases would fall outside of expected values. Perhaps with incorporation of an ROI tool, this hypothesis could be tested.

Although there have been numerous publications comparing various SWPs, scant literature evaluates accuracy or rMBF within transmural scar. Benz et al. reported the median rMBF in regions of scar was 0.68 mL/min/g [0.54–0.88] using PMOD software (Version 3.7; PMOD Technologies Ltd., Zurich, Switzerland) [33]. Although we did not specifically test PMOD, it has been shown to be highly correlated with 4DM and Cedars [13, 16]. As previously shown in Table 4, Seg-Scar in 4DM and Cedars rMBF (mL/min/g) was 0.71 [0.52–1.02] and 0.66 [0.51–0.85], respectively—findings that are nearly identical to that reported by Benz. Furthermore, also as shown in Table 4, median rMBF in normal tissue using 4DM, 4DM-FDV and Cedars was ~ 1.00–1.17 mL/min/g. Numerous other publications demonstrate rMBF in “normal” tissue using 4DM and Cedars to be 1.0–1.34 mL/min/g [13, 23, 25, 38]. The combination of these comparison findings substantiate that our methodology was accurate and not biased by user error or camera type.

4DM offers two options for MBF calculation using a 1-tissue compartment model (with and without an FDV). Interestingly, while Seg-Scar was significantly higher with 4DM than 4DM-FDV, whole heart rMBF was identical. To produce these findings, regions with normally perfused tissue were assigned much higher values (often unrealistically so) with FDV, as depicted in Figs. 8 and 10 and Table 4. Thus, although 4DM-FDV demonstrated less upward bias than 4DM within the region of infarct (i.e., 62% vs. 15%), rMBF outside of the infarct zone appears biased.

Comparison of resting MBF in 4DM versus 4DM-FDV. rMBF (mL/min/g) in the infarcted segments (white values) is 0.50 versus 0.30, respectively. This fact confirms accuracy within the infarcted segments with 4DM-FDV. However, outside of the infarcted segments, most rMBF are higher in the FDV model. The orange segments represent rMBF that are unrealistic in the 4DM-FDV model

As stated in the Methods section, the quality of the dynamic PET studies was deemed adequate both manually and by 2 separate automated processes. Hence, the input datasets were of good quality. As MBF = k * myocardial uptake/arterial input, there are 3 potential causes of upward bias: (1) invalid partial volume correction factors (one component of the variable “k”), (2) erroneous vendor-specific myocardial uptake processing (e.g., inaccurate motion corrections, inaccurate myocardial boundary selection, etc.), and (3) underestimated arterial input.

The methods for partial volume corrections for 4DM, Cedars and Emory-V software packages are not visible or modifiable by the user. Therefore, it is plausible that these SWP have overestimated the PV loss causing an upward bias in rMBF.

Another possibility for upward biased rMBF in TMS is software-specific erroneous myocardial uptake processing. All SWPs demonstrated nearly identical findings on relative perfusion (myocardial uptake) in terms of %RU and infarct size (all confirming visually obvious large, severe defects as demonstrated in Table 1 and Additional file 1: Table S4). However, we did find several examples of erroneous myocardial processing with 4DM and Cedars as demonstrated in Fig. 11. Due to the absence of myocardial uptake and LV aneurysmal segments within the TMS, the SWP inaccurately identified myocardial boundaries. This error could not be remedied manually with standard tools within the SWP. This erroneous boundary detection led to a “spillover effect,” yielding falsely elevated MBF values in the scar. This error did not occur with Emory-V, Emory-O or HS which implement both retention and compartmental kinetic models. Hence, we can conclude that in these situations, the upward bias was caused by vendor-specific errors in implementation of the kinetic model and not within the PET dataset. However, these examples were rare, occurring in less than 10% of the studies.

Inaccurate boundary selection. Due to the absence of myocardial uptake and LV aneurysmal segments within the transmural scar, the software package inaccurately identified myocardial boundaries, an error which could not be remedied manually with standard tools within the SWP. This erroneous boundary detection led to a “spillover effect,” yielding falsely elevated MBF values in the scar. Bottom left corner is an echocardiogram with the LV aneurysm outlined in pink

As very low MBF is proportional to the ratio of myocardial uptake to the arterial input function (tracer roll-off does not happen until higher flows), reduced %RU (as seen with the Cedars SWP) should yield lower MBF values. As this was not the case for 4DM, Cedars or Emory-V, the logical explanation for routinely upwardly biased MBF data is underestimated arterial input by the software package. In fact, this is the most logical explanation given the data we collected. Unfortunately, testing arterial input is technically challenging, not practical, and there are no standardized methods for doing so. Bui et al. [17] have demonstrated accuracy of arterial input with 2D and 3D scanners with the use of a phantom pump using a simplified retention model. However, testing of various software which utilize a compartmental model was not performed. Furthermore, it is hypothesized that a standard “one size fits all” location for the arterial input ROI (as used by 4DM, Cedars and Emory-V) is inadequate and individualized placement is necessary to achieve accuracy and consistency in perfusion metrics [17, 39].

Therefore, we can state that in a small proportion of cases in 4DM and Cedars, the upward bias of rMBF in TMS was due to inaccurate myocardial boundary detection. In the vast majority of cases in which upward bias was present, the cause could be due to erroneous PV correction, but the more likely cause is underestimated arterial input.

Finally, whole heart COV, and COV within scar and normal regions, was ~ 6–9% using HS, which is slightly lower but comparable to prior findings by Kitkungvan et al. with scans performed on a 2D scanner [40]. All SWP demonstrated COV ~ 10%, with the exception of 4DM (Table 4). Thus, same-day test/re-test methodologic precision has not changed significantly with the advent of contemporary 3D PET-CT systems.

If CAD outcomes (e.g., death and spontaneous MI) are modifiable with revascularization, we postulate that specific actionable thresholds of MBF must be met to achieve these goals. Thresholds have been identified and combined into a novel metric known as coronary flow capacity (CFC). Outcome data have demonstrated significant reduction in death and MI when CFC is used to select for revascularization versus medical therapy [41, 42]. We have demonstrated that artery-specific reduced CFC also predicts improved perfusion metrics after revascularization that is not realized with angiographically guided revascularization [43]. As such, researchers and clinicians will require standardization and accuracy across software platforms such these end goals can be achieved. However, in this study, we have demonstrated that not only do SWP differ in performance, but several SWP yield data that are not reliably accurate enough for clinical decision making.

Our study has some limitations. We did not attempt to determine the cause of SWPs’ inaccuracies, as this was outside of the scope of this manuscript. We hope that vendors gain insight from our findings and implement the necessary improvements. Secondly, all studies were processed by nuclear cardiologists who had been trained by the SWPs’ vendors. The studies were not sent to the vendors for processing or troubleshooting. Thus, it is possible that expert vendor staff could have improved implementation of their models on a case-by-case basis. However, as such service is not typically available, our methodology reflects real-world clinical practice. Each patient underwent 3 resting scans, and COV was  ~ 10% for all SWP. Thus, upward bias with associated narrow intra-software variability goes against user error and more likely correctly functioning (albeit erroneous) software. In this manuscript, we focused on “low flow” accuracy because it serves as an excellent reference with narrow margins. We did not assess “high flow” accuracy or “normal” volunteers in this study.