Trial Design

This study used a pragmatic randomized controlled trial design. Ethical clearance was provided from the Children’s Health Queensland Hospital and Health Service Human Research Ethics Committee (HREC/21/QCHQ/80217) and The University of Queensland (2021/HE002620).

Participants

Participants were recruited via an expression of interest (EOI) that was disseminated via email across existing professional networks. Written consent was obtained through the EOI. To be eligible to participate in the study, participants had to (a) be a qualified speech pathologist (i.e., not a student); (b) have potential to, or already be seeing children with pediatric feeding disorder; (c) work in Queensland, Australia; (d) have the capacity to attend the in-person simulation at the Queensland Children’s Hospital for the purposes of randomization; and (e) meet COVID-19 vaccination requirements for onsite visits at the research facility. During the EOI process, participants self-identified as either experienced (> 6 months experience with infant feeding) or novice (< 6 months experience with infant feeding).

Interventions

Participants were randomized to participate in either an in-person simulation or a telesimulation. The same simulated scenario was used for both modalities, with adaptations made for the telesimulation condition. The same facilitators were used for all sessions. The feasibility and acceptability of telesimulation using the same scenario was demonstrated in a previous study, using an iterative process of user testing and adaptation [18].

The simulation involved a 3-week old infant with laryngomalacia, and the session included five components: a pre-brief, didactic teaching, part-task activities, pause-discuss scenario, and debrief. All participants received the same pre-learning materials ahead of the simulation to support their preparation. Each group contained a maximum of six participants and included a mixture of self-identified novice and experienced participants. Sessions were targeted at the novice level learners and were delivered using the same content during each session. Further detail regarding simulation content can be found in a previous paper [18]. The simulations delivered in both conditions aligned with the Healthcare Simulation Standards of Best Practice [19].

For the in-person condition, participants attended the session on site at the Queensland Children’s Hospital, completing the simulation as a group over 4 h in a single room. Due to COVID-19 restrictions, social distancing was applied, and face masks were worn during the sessions. Due to their in-person presence, participants were able to manually handle the mannequins and equipment (e.g., bottle nipples) during the part-task activities.

For the telesimulation condition, participants were sent a Zoom® link ahead of their scheduled 4-hour session. It was requested that they link in individually from a quiet room using a computer with audiovisual capacity. They were asked to have a doll or plush toy available at the time of the simulation and used this to practice oral reflex examination and positioning during the part-task activities (as a substitute for the mannequin used in the in-person simulation). Exploration of different bottle nipples was done visually using shared images over Zoom®.

Outcomes

Each participant was assigned a unique identifier for use throughout the study, to allow for outcomes collected at different points to be compared. Outcomes were collected across three different time points: pre-simulation, immediately post-simulation, and at 4-weeks post-simulation. The primary outcome was clinical reasoning, with secondary outcomes as self-perceived confidence and anxiety, and satisfaction.

Demographic Questionnaire

At the commencement of the study, a demographic questionnaire was used to collect information regarding each participant’s workplace context, and their previous experiences and/or training in infant feeding. The questionnaire included 16 questions (13 multiple choice, and 3 open-ended), and was adapted from a previous study [20].

Clinical Reasoning

A clinical vignette was used to assess change in clinical reasoning skills pre- and post-simulation. A written case scenario was presented to the participants, and they were asked to write everything they would do with the patient at that time, including any clinical tasks, future planning, considerations for family-centred care, and onward referrals. The same vignette was presented at the pre- and immediately post- timepoints, but a different vignette was presented at the 4-week post timepoint. The 4-week post vignette was matched to the original vignette according to approximate age of the case, and number of significant features with regards to family situation, relevant co-morbidities, and feeding context cues. The purpose of the new vignette was to assess whether participants could transfer their learnt skills to a new case. The clarity and authenticity of the clinical vignettes was checked by three expert clinicians at the Queensland Children’s Hospital, and ideal answers were developed.

A marking rubric was designed with precedence from another study regarding pediatric feeding simulation [17]. The clinical vignettes were marked against five criteria: clinical tasks (i.e., What do I need to do?), clinical reasoning (i.e., Why do I need to do the tasks I selected?), planning (i.e., Did I consider the bigger picture and what might happen next?), family-centred care (i.e., Did I involve the family/patient in my decision-making?), and interprofessional care (i.e., What is my job? What did I need to ask someone else to do? Did I ask the right person?). Each criteria included four levels of attainment: minimal, developing, competent, and mastery. Clear instructions for meeting each level were developed, leading to a total possible score of 20. A copy of the clinical vignettes and the marking rubric is available as supplementary material (Appendix A). Each vignette was then piloted with three novice clinicians at the Queensland Children’s Hospital to refine the marking criteria. Marking of the vignettes was completed independently by two members of the research team who were blinded to allocation and timepoint where possible (i.e., it was not possible to blind markers to the third timepoint, which used a different vignette). A third party reviewed any disagreements in scoring.

Self-perceived Confidence and Anxiety Questionnaire

Participants were asked to rate their confidence and anxiety with regards to infant feeding care at all three timepoints, using a questionnaire adapted from a previous study [21]. The questionnaire included two visual analogue scales, where participants were asked to rate their overall confidence and anxiety from 0 to 100, where 0 = not confident/not anxious at all, and 100 = very confident/extremely anxious. The questionnaire also included a series of statements where participants were asked to rate their agreement along a 5-point Likert scale (e.g., “I am confident in my ability to identify infants who need care for feeding disorders and/or dysphagia”). Participants’ perceptions of the impact of simulation on their confidence and anxiety were captured in two additional questions in the questionnaires completed post-simulation. An additional question at these timepoints also explored participants’ perceptions of the impact of specific simulation components.

Satisfaction

A satisfaction questionnaire was completed immediately post-simulation and was developed from a previous study [21]. This questionnaire included six statements, and participants were asked to indicate their agreement with each statement along a 5-point Likert scale (e.g., “Involvement in the simulated learning experience was a worthwhile experience for me”). Participants were also asked to rate their satisfaction with each simulation component.

Sample Size

The extent of reasonable change was estimated using the scoring of the primary outcome measure of clinical reasoning from a previous paper, where a change score of 2.0 ± 1.5 points for the overall rating on the clinical vignettes was considered clinically important [17]. Power calculations indicated that a sample size of 16 participants per group was required in order to detect a statistically significant change of at least 2.0 points in clinical reasoning scores (power = 0.90, α = 0.01; d = 0.8; nQuery version 9.3.1).

Randomization

Block randomization (2 novice to 1 experienced) was used to allocate participants to their respective simulation modality to achieve a balance of experience levels across modalities. This ratio was determined according to the number of experienced vs. novice participants who completed an expression of interest. Including participants with different experience levels in the simulation was identified as valuable in our feasibility study [18]. A computer-generated random number sequence was used to generate the allocations. Allocation was completed using concealed envelopes by a neutral party external to the study.

Blinding

Due to the nature of the program provided, it was not possible to blind the researchers providing the simulations, nor was it possible to blind the participants to their allocated modality.

Fidelity

Fidelity checks were completed at the end of each simulation session that included descriptive details regarding the number of participants who attended, the ratio of novice to experienced participants, and the structure of the session, to ensure that the simulations provided were the same across groups. Details regarding structure of the session included whether the essential elements of simulation were provided, comprising a pre-brief, didactic, part-task activities regarding respiratory distress, vital signs identification, and infant feeding management, and the simulation itself. Information captured regarding the simulation itself included the use of visual elements (backgrounds, costumes, and cognitive aids), whether modelling from an experienced clinician was provided, and whether the two facilitator roles were kept as distinct and separate during the simulation itself. Finally, the length of the debrief provided at the end of each session was recorded.

Statistical Analyses

The Statistical Package for the Social Sciences (SPSS, version 27) was used for data analysis. Data were presented predominantly using descriptive statistics. In seven cases, the final 4-week post datapoint was missing. In order to prevent bias via the exclusion of missing data, data was imputed using the missing participants’ previous scores [22]. Where data were not normally distributed, a transformation using the square root of the data was completed, and normality confirmed [23]. This was conducted for the variable of overall confidence. Groups were compared at baseline using a chi-square test to examine differences in proportions, and an independent t-test where data were continuous and normally distributed.

Inferential statistics were used to explore the primary outcome measure of clinical reasoning, as measured by the overall scores on the clinical vignettes. Differences between groups and across timepoints were assessed using a two-way mixed analysis of variance (ANOVA) following confirmation of normal distribution using the Kolmogorov-Smirnov test. This testing was repeated for the variables of overall confidence and anxiety ratings, using the transformed data for confidence ratings to ensure normal distribution. No assumptions were violated and there were no significant outliers in the data. Ordinal data in the individual confidence statement ratings were compared across timepoints using a Friedman test, and a Mann-Whitney U test to examine specific differences according to randomization and experience levels. Satisfaction data between groups were compared using a t-test, following confirmation of normal distribution. To control for the testing of multiple hypotheses, a conservative p-value of 0.01 was applied.