Microcirculatory monitoring has not yet been clinically adopted despite more than a decade of research and technological advances. Offline manual analysis is time consuming and occurs away from the patient. To expedite translation into the intensive care unit, several corporate entities have attempted to develop automated analysis software. Studies examining microcirculatory function in critically ill cohorts using automated analysis have reported contradictory findings to the wider literature [18]. The potentially confounding effect of unreliable automated analysis is a significant limitation of such studies.

Examination of microcirculatory function in various pathologic states is an active area of research. Until recently, analysis of microcirculatory clips was undertaken using AVA 3.2, which employs a semi-automated approach involving manual tracing of vessels and designating a semi-quantitative flow metric to each vessel. To improve time efficiency, more recent versions of the software (AVA 4.3C) now undertake vessel segmentation and flow analysis automatically. Poor reliability of automated analysis (AVA4x) has been demonstrated in several contexts [13, 14]. Presently, therefore, automated analysis using versions of AVA 4 is inappropriate for research use and clips must continue to be analysed in AVA 3.2 [13].

Further complicating SDF clip analysis is that the current generation of Microscan cameras record in a different resolution (1280 × 960) to that which is compatible with analysis in AVA 3.2 (640 × 480). We have therefore described an approach whereby clips are recorded using current generation cameras (as older generation cameras are no longer available), resized in ImageJ and imported into AVA 3.2 for manual analysis. In this study we have examined the effect of resizing clips on MIQS, and, the interobserver reliability of this process and the data generated from analysis. Finally, we examined agreement between AVA 4.3C and AVA 3.2 in analysing resized clips.

Current guidelines for microcirculatory assessment outline several image characteristics that impact analysis result but are unrelated to the underlying physiology of the patient. These include illumination, duration, focus, content, stability, and pressure. Errors in any of these parameters compromise the validity of the image for analysis. The MIQS quantifies these parameters to confirm that clips are appropriate for analysis; clips failing at this stage are discarded. Clips captured using AVA 4.3C were resized in ImageJ for analysis in AVA 3.2. We found that reducing the resolution of the clip did not significantly change the MIQS assigned by an experienced operator. This has not been previously demonstrated in the literature, however is an expected finding when considering the domains examined by the MIQS.

The MIQS score has been employed in the current study to confirm that changing resolution does not compromise suitability for analysis (mainly illumination and focus), not as an indicator of change in the analysis readout—MIQS scores were assigned before any analysis. Though it is unlikely resizing in ImageJ would change the MIQS, it is important that resized clips are evaluated before manual analysis to confirm MIQS scores are of sufficient quality. Indeed, we found that resizing the clips in ImageJ does not impact the MIQS score.

Resized clips were then imported into AVA 3.2 for semi-automated analysis by two experienced operators. A positive bias between observers was demonstrated for TVD, PVD and HI; while a negative bias was demonstrated for PPV and MFI. Regression analyses for each parameter were non-significant, indicating that bias between observers was not proportional to mean value of the parameter. The magnitude of the biases (difference between observers) is problematic when considered in the context of previous work in critically ill cohorts. For example, in the MICROSHOCK study [19], the difference in PVD between trauma patients who did and did not develop organ dysfunction after injury was approximately 2.6, whereas bias between observers in our study was approximately 6.3. Based on these data, analyses undertaken by separate observers should be compared acknowledging that biases between observers are likely greater than the clinically significant difference in parameters. Analysis for a particular study should therefore be undertaken by a single investigator. Though interobserver reliability for AVA 3.2 has not been previously examined, Scheuzger et al. [20] demonstrated good interrater reliability for manual analysis using CytoCam Tools. Cytocam Tools is produced by a different manufacturer, however, generates similar microcirculatory parameters to AVA 3.2. It should also be noted that Cytocam Tools is used to analyse incident darkfield (IDF) images (rather than SDF as used here) which may impact analysis reliability.

Finally, we examined the agreement between manual analysis (AVA 3.2) of resized clips and the automated analysis (AVA 4.3C) of the original clip. Significant proportional bias was observed for PVD and PPV between the two analysis methods. Poor agreement between automated and manual analysis has been previously reported [13].

We have demonstrated poor interobserver reliability between automated analysis in AVA 4.3C and manual analysis in AVA 3.2. Whether the differences between the two approaches is clinically significant remains to be seen. It is conceivable that automated analysis may be of clinical utility if it is able to effectively detect a change in microcirculatory function over time, even if the absolute values calculated deviate significantly from gold standard manual analysis. Assuming manual analysis has reasonable intraobserver reliability (as previously reported) [21], reliable detection of change in microcirculatory function by automated systems would require a consistent difference between automated and manual analysis for high and low values of microcirculatory parameters. We have demonstrated in this work that the magnitude of difference between automated and manual analysis changes with the value of the microcirculatory parameter. Specifically, for PVD a significant positive regression coefficient was evident on Bland–Altman analysis. In practice this suggests that for lower values of PVD analysis techniques had greater agreement than at higher values of PVD. The change in magnitude of deviation from the gold standard manual analysis across a spectrum of PVD values suggests that automated analysis would be unreliable in detecting deterioration of microcirculatory function in a patient.

The ability of automated analysis to detect changes over time has not been formally analysed and represents an important future direction from this work. The existing literature is variable with some studies reporting significant improvements in microcirculatory function with time [18] while others suggest that automated analysis by AVA4.1 (an earlier iteration of AVA4.3C) is insensitive to changes in microcirculatory function associated with a changing clinical state [13]. A relevant future study would involve serial imaging of the sublingual microcirculation in the intensive care unit with analysis of clips by automated software and manual techniques. A comparison could then be made between automated and manual analysis for reliability in detecting changes in microcirculatory function over time and predicting clinical outcome.

Current generation cameras that record in higher resolution produce images that improve the visible contrast between red blood cells in the microcirculation and surrounding stroma. The problem, as identified quantitatively in the present study, is not that higher resolution cameras are unreliable but that the automated analysis software that is coupled with higher resolution systems is unreliable. Manual analysis of microcirculatory clips therefore remains the gold standard. This is evidenced in the current guidelines for microcirculatory assessment [11] and supported by a dearth of high-quality evidence for a shift to automated analysis. Given that manual analysis requires lower resolution clips (software compatibility) than are recorded by current generation cameras, and, that older generation cameras are no longer available; it appears that the only option is to reduce the resolution of clips for manual analysis as reported here.

Until reliable automated analysis for current recording parameters is available or manual analysis software is updated for compatibility with higher resolution clips, increasing the resolution of cameras offers little benefit to clinical monitoring of the microcirculation. It does, however, appear that reliable automated analysis is approaching successful implementation. Once adopted more broadly, these higher resolution cameras will be of use. Our recommendation, based on the current study and the landscape of automated analysis approaches, is that clips be recorded using current generation technology in preparation for future automated analysis but continue to be manually analysed (using methods reported here) until automated protocols are validated.

Given the issues around reliability of current automated analysis, alternative analysis approaches are required that provide both reproducible and timely results. Microtools is a validated computer vision algorithm developed by Hilty et al. [22] for microcirculatory clip analysis. Microtools has been validated against semi-automated analysis in clinical populations and animal models [22, 23]. Currently, Microtools has been validated using IDF videomicroscopy clips. However, it represents a future avenue for analysis of SDF clips when validated. The developers of the AVA software have also recently released an updated version of the software, AVA 5, however this version remains to be validated (or reported) in the literature.

This current study has several key limitations. Firstly, we have examined clips from healthy volunteers. Though unlikely, it is possible that the current reliability data is not translatable to microcirculatory samples collected in pathologic states. Expansion of the current study population to include patients across a range of pathologies would therefore be of interest.

Enrolment of exclusively female participants in the current study may limit generalisation of reported microcirculatory parameters. Similarity in microcirculatory parameters between healthy males and females have, however, been previously reported [24]. The clips analysed in this study are derived from a database of human clips analysed in previous work comparing sublingual microcirculatory parameters in anaesthetised pigs to adult human [25]. This previous study was matched (female only) to reduce potential sex-related variation in microcirculatory parameters. Importantly, the purpose of this study was to examine interobserver reliability, rather than define normal ranges for microcirculatory parameters in healthy adults. We therefore suggest that the enrolment of exclusively female participants would have very limited (if any) effect on the interobserver reliability data reported here.

Though we have demonstrated interobserver reliability in this study; intraobserver reliability was not examined. High intraobserver reliability (> 0.8) for TVD, PVD and PPV has been demonstrated by repeated semi-automated analysis of SDF clips in AVA 3.0 (an earlier version of AVA 3.2 that involves a similar semi-automated analysis process) [21]. Given the similarities between these generations of AVA software, we expect the findings of Peterson et al. [21] to be generalisable to this work. Nonetheless, the current study would benefit from repeat analysis of our suite of resized clips. Finally, both observers in the current study were of similar experience levels. Examination of reliability between experienced and novice observers would be of interest with relevance to clinical utility of these tools.