Between January 2020 and January 2021, we invited a convenience sample of 14 internal medicine residents with any prior lung ultrasound (LUS) training to participate in this cross-sectional study. We excluded those with no prior LUS training as tracking uninformed eye movements during image interpretation may not yield helpful information.

After performing eye-tracking calibration in a seated position, consenting participants viewed and interpreted 8 LUS videos on a standardized laptop (Asus ROG Strix, GL503V) with an eye-tracking system (Tobii Tech, Danderyd, Sweden) mounted on the laptop. Eight videos were created from 6 s cineloops from our program’s anonymized teaching bank. These cineloops were played in a continuous loop for 30 s, portraying the following common LUS findings: normal lung (× 2), absent lung sliding, pleural effusion, mirror image artifact with a negative spine sign, pleural irregularity with B-lines, M-mode demonstrating absent lung sliding, and presence of B-lines and A-lines. Each video is accompanied by 1–3 questions regarding the findings and diagnosis (See Additional file 1). Participants were instructed to read the paper-based questions for each video prior to viewing the video, so that they are aware of what findings to anticipate. Participants had the option to exit the video early or view the video one additional time, within 1 min (max allotted duration 2 min).

AOIs for each ultrasound video were defined as areas on the ultrasound image that required evaluation to rule in or rule out a specific finding. For example, evaluation of the spine in the far field of a coronal image of the lung base is important to rule in or rule out a pleural effusion and examination of the pleural line is important to determine if pleural sliding is present [18]. AOIs for each video were mapped independently in March 2021 by two experts (IM, JD), both certified by the American Registry for Diagnostic Medical Sonography. Three discrepancies in AOI mapping were resolved by discussion. One discrepancy involved evaluation of the lung zone labelling for a normal lung cineloop and a second discrepancy involved evaluating the depth scale in a cineloop for B lines. Post discussion, experts agreed that neither were definitively critical to the cineloops’ diagnosis. The third discrepancy involved evaluation of the far field findings deep to a non-sliding pleura, which was agreed upon to be an important area to evaluate. Both experts were blinded to the participant data, which was collected by the resident investigator (ML). The AOI were then externally validated using eye movement data of a third expert (ACT) external to our institution, whose eye movements were captured in September 2019 during a site visit.

Total fixation duration was defined as the duration of all fixations within the AOI, (I-VT filter, default settings, minimum fixation duration of 60 ms, User’s manual Tobii Studio, version 3.4.8, 2017, pp. 54–57). Total time spent viewing the videos was time spent both within and outside of AOI. Gaze plots were created using Tobii Studio software and examined qualitatively (Fig. 1).

Gaze plots of expert (green, top left) and novices (purple) in identifying the presence of a positive spine sign and the pleural effusion. Top right: gaze plot of a novice who scored 0% on the video with minimal gaze on the spine. Bottom left: gaze plot of a different novice who scored 50% on the video. Bottom right: gaze plot of a third novice who correctly identified both findings and scored 100%

Accuracy score was calculated as the number of correct responses on image interpretation (out of 15), presented as a percentage. Validity evidence for the questionnaire was evaluated in two ways. First, in September 2019, for content validity, the questionnaire was reviewed and completed independently by two education experts (JD, ACT) not involved in test construction; both scored 100%. Second, we evaluated the internal reliability of the questionnaire using Cronbach’s alpha (alpha = 0.68).

Standard descriptive statistics are reported. The independent association between eye-tracking variables and accuracy score was explored using univariate linear regression analyses. A two-sided p value < 0.05 was considered to indicate statistical significance. All analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC) and STATA 17.0 (StataCorp, College Station, TX).