Group-level learning curves average results across a cohort and provide useful information for educators to guide education interventions and assessments [1]. Plateaus on group-level learning curves represent where a cohort of learners, on average, are unlikely to improve further unless an educational intervention is applied. Despite their utility, group-level learning curves do not represent the learning of any particular individual, and individuals might benefit from further repetition beyond a group-level plateau [1]. As such, this paper avoids equating plateau points with competence. Rather, understanding typical learning curves and plateaus helps guide ideal timing for re-education for those needing to improve performance [1]. Additionally, the learning curves help guide the appropriate timing of holistic learner evaluations, such as an observed structured clinical encounter in education or workplace competency assessment in clinical practice, by organ system to determine which students have competency and which require further assistance [1]. Importantly, such an assessment would require assessment of both the psychomotor skills of image generation and the cognitive skills of image interpretation and clinical integration.

This data suggest that learning curves differ for specific POCUS applications, which is corroborated by prior literature [7,8,9]. Our data suggest that bladder POCUS has the quickest learning curve for novice learners. Abdominal aorta, lung, and renal exams have longer learning curves, that overall similar to each other (plateau points 19, 16, and 15, respectively). Cardiac POCUS had the longest learning curve, without a plateau point for overall image quality, suggesting ongoing improvement is noted beyond 25 examinations.

To our knowledge, learning curves for bladder ultrasound have not been previously defined. Bladder POCUS had the highest rate of exemplary image acquisition scores throughout the entire curriculum, with performance remaining stable over the course of the curriculum. This data suggest that bladder POCUS skill can be acquired rapidly for novice learners.

Data regarding learning curves for abdominal aorta POCUS demonstrate substantial heterogeneity. In assessment of military physicians, it was thought that 20 examinations were sufficient for a plateau of learning [12]. However, in a more robust analysis of emergency medicine physicians, Blehar demonstrated a plateau point of 84 examinations [7]. Importantly, in the study by Blehar et al. “learner image quality was assessed by comparing their image quality to the image quality of the expert reviewers’ independent imaging performed during the study”. Comparison to experts in the Blehar et al. likely prolonged the learning curves, where our data suggest that learning based on a standardized assessment plateaus much sooner. Similarly, data regarding lung POCUS have substantial variation. Blehar et al. found a plateau point of 39 lung examinations, but most other studies favor a steeper learning curve for lung ultrasound [7, 12, 14, 15]. Our study corroborates that psychomotor skill acquisition for the lung plateaus relatively quickly for novice learners.

Review of the literature shows that plateau points for cardiac POCUS occur after between 20 and 36 examinations [7,8,9, 21]. Importantly, Millington found a plateau point of 20 examinations in surgical residents with “low experience” with POCUS. However, residents were defined as being “low experience” despite completing up to 25 examinations prior to the study [9]. This likely represents an underestimation of true learning curves, given the learners’ prior experience. Blehar noticed a statistically defined plateau point after 27 cardiac examinations, however, on graphical representation improvement is continually noted after this point [7]. While assessing “entrustability” of POCUS skills, Clunie found that predefined entrustability was met after an average of 36 exams [8]. Our study would suggest that cardiac POCUS is the most difficult application of those studied within this cohort to learn, and that learning might continue beyond 25 examinations. Educational and evaluation measures to ensure adequate learning should be designed with this knowledge in mind.

Depth, tomographic axis, and gain are important components of image quality [4]. Analysis of rates of scores (1–5 on the Likert score) and learning curves for depth, tomographic axis, and gain demonstrates that tomographic axis has the longest learner curve of the three components of image quality for this cohort of learners. Visually, the components of depth and gain were noted to have higher rates of exemplary scores earlier on in learning than when compared to axis (supplemental digital content 3–7). This is corroborated by learning curves, within specific organs. For example, statistical plateau points for abdominal aorta were noted at 19 for overall image quality, 14 for depth, and 15 for gain; however, ongoing improvement without plateau was noted for tomographic axis. Regarding cardiac imaging, the combination of higher rates of exemplary scores on depth and gain, combined with a lack of statistical improvement over time suggest students were able to rapidly acquire these skills. However, data for cardiac axis demonstrate worse performance overall, and a plateau point at 17 examinations. This would suggest that tomographic axis was the hardest to master. Overall, this would suggest that tomographic axis was the hardest concept of the three for this cohort of learners to grasp. One potential conclusion is that the concepts of depth and gain are more generalizable between organ systems, while an individual organ’s tomographic axis is unique. This difference between components of image quality has not been reported in the literature previously. While teaching all principles of image quality is important, this data suggest that curricular changes paying particular attention to optimizing the unique tomographic axis for each organ system might benefit our program. Further research is required to see if this is finding can be generalized to other learners.

Limitations to this study include it being from a single cohort of students within a specific educational program. While students were encouraged to save all examinations within their Butterfly Network archive, it is likely that some studies over the course of the curriculum were not saved by students, and thus not reviewed. This potentially impacts learning curves and plateau points, however, given the requirements of the students to complete a portfolio of images it is unlikely that this would represent a significant number of missed examinations. While all students completed the required minimum number of exams, incomplete capture of data is noted during the image review and data exportation phases of the research project, as some students had fewer than 25 examinations per organ system in the extracted data. We believe this represents a small subset of students, with minimal impact on overall data analysis. Lastly, while practically there is benefit of learning the psychomotor skill component on healthy individuals, this study does not address learning curves on, or transfer of skill to actual patients in the clinical environment.

Further research incorporating multiple sites and educational programs is needed to identify more generalizable data. It stands to reason that the quality and amount of education on a specific topic will affect learning curves. However, studies evaluating the impact of instructional design on learning curve do not exist currently. Studies assessing the effect on learning curves of the amount of education, the type of evaluation, and the frequency of feedback should be completed. Additionally, research to assess transfer of skills from practicing on healthy individuals to actual patients is required to determine optimal amounts of practice in preclinical education.