1 INTRODUCTION

The ongoing global pandemic caused by a novel strain of coronavirus (SARS-CoV-2) has severely impacted the delivery of routine medical care globally. Beginning in mid-March 2020, leading healthcare organisations and governments around the world recommended the immediate postponement of all elective operative cases (Iacobucci, 2020). Many in-person clinic visits for all medical and surgical specialists transitioned to telehealth platforms. Further, many patients needing care for non-COVID-19 related concerns, including musculoskeletal issues, have been unable or unwilling—understandably due to fear—to receive it. Indeed, a report by the Commonwealth Fund demonstrated a 61% reduction in orthopaedic clinic visits through mid-March 2020 (Mehrotra et al., 2020). Further, nearly 30% of all visits to ambulatory practices are now provided via telemedicine (Mehrotra et al., 2020). These ‘invisible patients’ suffer (Thomas H. Lee, 2020), and as the world begins to fully ‘reopen’, we must strategise how best to optimise the delivery of backlogged care and learn from this global pandemic moving forward. This is especially true in fields such as orthopaedic surgery (Jain et al., 2020).

Because of the notable backlog of patients needing orthopaedic care, there is likely to be a large stress on the health care system as it seeks to ‘catch up’. Most importantly, it is crucial to identify the patients who elected or were requested to forgo musculoskeletal care during the initial peak of the pandemic but are in true need of more acute care now. Thus, it would be of value to identify a simple, yet effective tool to triage ambulatory orthopaedic patients effectively. One possible instrument that may be of value in such a situation is a digital medical history (DMH), or summary of health information and concerns, completed by patients prior to receiving and/or attending an in-person clinic visit.

In the present study, we aimed to assess the initial results of our implementation of a DMH system as a quality improvement project at a single teaching hospital as a means of managing the spike in post-COVID-19 orthopaedic care requests. Specifically, we reviewed the prospective implementation of a rapid cycle, semi-randomised A/B testing approach at the patient level on our efforts to improve patient response rates. Such an approach can improve efficiency and quality within health systems by abandoning poor practices earlier on (Horwitz et al., 2019). We hypothesised that directly calling patients over the telephone would lead to the highest rate of DMH completion. Further, we suspected that positive email messages and early evening contact would lead to greater DMH completion rates. Last, we believed that increasing patient age would be associated with an overall decrease DMH completion regardless of contact technique.

2 MATERIALS AND METHODS

A questionnaire with pre-set questions and provided response options was designed to obtain a structured musculoskeletal medical history based on the current chief orthopaedic complaint, as well as prior relevant medical history. To begin, patients completed general health questions. Subsequently, each patient was guided towards completing questions regarding the reason for their desire to seek orthopaedic specialty care, including previous medical treatments and/or interventions and patient-reported outcomes measures (PROMs). Ultimately, the number of questions each patient completed depended on their chief musculoskeletal complaint. Prior to administration of the questionnaires, experienced orthopaedic surgeons and related staff reviewed the questionnaires and process internally, creating iterations over time prior to initial use. Indeed, the idea of using a DMH had been broached prior to its necessity during the COVID-19 global pandemic, so it had begun the thorough vetting process.

In total, four different rapid cycles occurred.

2.1 Cycle 1

Cycle 1 included all patients (n = 337) with orthopaedic surgery clinic appointments that had been scheduled but subsequently cancelled because of the COVID-19 global pandemic. These patients were further split into two groups based whether they had activated their EPIC patient portals (and provided their telephone numbers and email addresses) or not. In total, 166 patients had active EPIC patient portals (Group 1), while 171 patients did not (Group 2). For those with active EPIC patient portals, patients received the DMH and a message with completion instructions through this channel. After two weeks, those in Group 1 who had not responded via their EPIC patient portal were further split into being contacted by telephone or email (Group 1A: telephone; Group 1B: email). Patients in Group 2 were first contacted via letter by mail with clear instructions to activate their EPIC patient portal and complete the DMH in a timely manner. All patients of Group 2 not responding within two weeks to the initial letter were contacted by telephone. For all patients, the importance of completing the DMH was explained so that accurate triaging could take place and consultation by telephone could be scheduled with orthopaedic surgeons to inform them about conservative treatment options during the global pandemic or if further care or assessment was indicated.

2.2 Cycle 2

Cycle 2 included patients who were referred to the orthopaedic surgery clinic just prior to the start of the COVID-19 crisis but had not been previously scheduled (n = 138). This formed Group 3 of patients in the present study. A simple semi-randomised, A/B test was done with Group 3 to determine whether a positively-worded email led to higher rates of DMH completion compared to a neutrally worded email. Indeed, patients with surnames beginning with A through K received a positively-worded email highlighting the benefits of completing the DMH (n = 70) (Group 3A). Patients with surnames beginning with L through Z received a generic email simply asking them to complete the DMH (n = 68) (Group 3B).

2.3 Cycle 3

Similar to Cycle 2, patients in Cycle 3 included patients who were referred to the orthopaedic surgery clinic just prior to the start of the COVID-19 crisis but had not been previously scheduled (n = 84). This formed Group 4 of patients in the present study. A simple semi-randomised, A/B test was done with Group 4 to determine whether email timing impacted the rate of DMH completion. Patients with surnames beginning with A through Hi received an email at about 8:00 AM requesting completion of the DMH (n = 42) (Group 4A) and with surnames beginning with Ho through Z about 4:30 PM requesting completion of the DMH (n = 42) (Group 4B).

2.4 Cycle 4

Cycle 4 included all patients who were new referrals to the orthopaedic surgery clinic during the COVID-19 global pandemic and called the hospital to schedule an appointment (n = 81). This created Group 5 in the present study. When each patient reached the clinic via telephone, they were informed of the need to complete the DMH as well.

2.5 Ethics, funding and potential conflict of interest

The present rapid cycle, semi-randomised A/B testing approach observational study was exempt from institutional review board (IRB) approval because this work is an ongoing quality improvement project in the context of routine clinical care. Further, we assessed DMH response rates and did not utilize identifiable patient health data/protected health information (PHI). The study received no specific grant funding. The authors declare no related conflicts of interest.

2.6 Statistics

Across all patients and for each approach individually, the completion rate was calculated. Patient age (years) and anatomic injury location leading to the individual seeking orthopaedic care was recorded. Time to completion (days) was determined across all patients completing the DMH. Multivariable logistic regression was used to assess whether age (years) or anatomic injury location was associated with DMH completion. All statistical analyses were performed using Stata/SE 14.2 for Mac (StataCorp). A priori, significance was set at p < 0.05.

3 RESULTS

Figure 1 shows the flow chart of patient recruitment and assignment to the different approaches. Across all contacted patients, the DMH completion rate was 48% (307 out of 640 possible patients) (Table 1). Across all possible patients (except for six patients who did not have their age recorded), the average age was 53 years (SD: 18 years). In addition, the most common anatomic location of musculoskeletal concern was the knee (n = 164 [26%]).

An illustration of the rapid cycle, semi-randomised A/B testing approach used in this observational study. The breakdown of the number of patients by cycle and group are presented as well

TABLE 1. Overall patient Characteristics (n = 640) Characteristic Overall n (%) or mean (SD) DMH completion Yes 307 (48) No 333 (52) Age (years) (n = 634) 53 (18) Anatomic location Knee 164 (26) Foot and ankle 115 (18) Shoulder 90 (14) Back 100 (16) Hip 83 (13) Hand and wrist 17 (2.7) Elbow 10 (1.6) Non-specific/other 61 (9.5)

In Group 1, there were originally 166 patients. In total, 23 patients responded to the initial EPIC patient portal communication and completed the DMH. The remaining 143 patients were then split into two groups as outlined above. The completion rate of those contacted by email was 54% (43 out of 80 patients), while the completion rate of those contacted by telephone call was 32% (20 out of 63 patients).

In Group 2, there were originally 171 patients, and each patient was initially contacted via letter. At this point, a total of four patients (2.3%) completed the DMH. Overall, 113 patients were then able to be successfully contacted via telephone and reminded to complete the DMH. From those who were reached via the telephone, 28% of patients (32 out of 113 patients).

In Group 3, a total of 138 patients who were not previously scheduled for an appointment but were referred to the orthopaedic surgery clinic were identified. After splitting the patient sample as outlined above, 60% of patients (42 out of 70 patients) who received a positively worded email completed the DMH. Similarly, 66% of patients (45 out of 68 patients) who received a neutrally worded email completed the DMH.

In Group 4, the timing of a standard email requesting DMH completion was assessed. The completion rate for those receiving an email at 0830 was 26% (11 out of 42 patients). The completion rate for those receiving an email at 1630 was 57% (24 out of 42 patients).

In Group 5%, 78% of patients (63 out of 81 patients) who were referred to the orthopaedic surgery clinic and called during the COVID-19 global pandemic to schedule an appointment completed the DMH.

Across all cycles and groups, increasing age was associated with decreased odds of DMH completion (odds ratio [OR]: 0.985 (95% CI: 0.976–0.995), p = 0.002) (Table 2). Further, patients with back concerns (OR: 0.395 (95% CI: 0.234–0.666), p = 0.001) or non-specific/other musculoskeletal concerns (OR: 0.331 (95% CI: 0.176–0.623), p = 0.001) had decreased odds of completing a DMH.

TABLE 2. Factors associated with DMH completion across all cycles and groups Pseudo R-Squared 0.038 Characteristic Odds ratio 95% Confidence interval p-Value Age (years) 0.985 (0.976–0.995) 0.002 Injury location Knee Reference Foot and ankle 0.835 0.514–1.356 0.466 Shoulder 0.974 0.578–1.643 0.922 Back 0.395 0.234–0.666 0.001 Hip 0.845 0.488–1.460 0.546 Hand and wrist 1.858 0.620–5.568 0.268 Elbow 0.445 0.120–1.653 0.227 Non-specific/other 0.331 0.176–0.623 0.001 4 DISCUSSION

The COVID-19 global pandemic provided an opportunity to drive forward innovative solutions in musculoskeletal medicine that may leave a positive lasting impact on care delivery. Specifically, the use of technology to both provide care via telemedicine (Bernstein et al., 2020), as well as efficiently assess and triage patients, is one area with a great deal of work underway. In the present study, we found that around 50% of patients completed a DMH when contacted to do so. Unsurprisingly, those patients who took the initiative to call once a referral had been placed also completed the DMH at the highest rate. Unfortunately, increasing age was associated with decreased odds of DMH completion, though only minimally so.

Our study has a number of notable strengths. First, the current global pandemic allowed for a rare natural experiment environment in which patients were required to engage with the orthopaedic clinic using technology. Second, our rapid cycle, semi-randomised A/B testing approach to this quality improvement project allowed for an easily understandable and comparable set of findings over a relatively short period of time. Such insight can not only guide clinical care but more in-depth study in this area in an efficient manner. Third, this assessment only included patients seeking clinic-level musculoskeletal care, reducing the unmeasured variation of the patient sample. Thus, while there was variation in patient concern leading to seeking an appointment (e.g., hand injury vs. knee injury), patients needing emergent care were not included in this study. Lastly, the focus of this quality improvement project may be directly related to the COVID-19 global pandemic, but the findings could have a lasting, positive impact on musculoskeletal care moving forward.

While there are certainly strengths this work, there are also a number of limitations that readers should keep in mind when assessing this study. First, the wide breadth of musculoskeletal clinical concerns was not accounted for in this study. Patients who felt that their concerns were more urgent may have been more apt to complete the DMH in order to be seen. Because this was not controlled for a priori, we cannot confirm that our findings are not biased by this issue. Second, health literacy or other sociodemographic-related factors may be associated with successful DMH completion and/or the mode of communication. Unfortunately, such information is not available in our data set. Further study is warranted in this area that incorporates these important variables. Third, we utilised a semi-randomised approach instead of a true randomised approach to ensure feasibility of this project, as well as to ensure that we were nimble enough to adjust as needed. We believe this is an acceptable methodological approach given the natural experiment setting of the COVID-19 global pandemic. Fourth, we are unsure if our findings are generalisable—both in musculoskeletal clinics across the Netherlands, as well as around the world. Patient expectations, engagement, and differences in the structure of health systems may play a role in DMH response rates that we were unable to address in the present study. Lastly, our study may be underpowered; thus, the risk of Type II error is increased. However, we are limited by the available patients given the natural experiment setting of this quality improvement project.

Across the world, the COVID-19 global pandemic has forced orthopaedic surgeons and their patients to adapt and innovate rapidly in order to deliver and receive appropriate and necessary musculoskeletal care. This has led to the widespread adoption of new or updated technology and processes (Bini et al., 2020). For example, telemedicine has become a routine component of orthopaedic care delivery, and patients and surgeons tend to be satisfied with telehealth encounters (Rizzi et al., 2020). Further, treatment plans devised via telemedicine have been shown to rarely change following in-person preoperative or pre-preprocedural encounters, suggesting that such technological integration helps make care delivery more efficient for all parties (Crawford et al., 2021a, 2012b; Lightsey et al., 2021). In addition to the exponential growth of telehealth encounters, other areas of advancement that show promise or have been emphasised during this crisis are the collection of patient-reported outcome measures (PROMs), and, in the present study, the collection of a DMH. Regarding PROMs, it is already known via prior research that having patients complete PROMs via email or other web-based platforms can be an efficient way to routinely collect PROMs with less resource utilisation compared to telephone or traditional mail approaches (Schwartzenberger et al., 2017). However, other evidence suggests that PROMs response rates via email is lower than when PROMs are asked to be completed via telephone or traditional mail (Palmen et al., 2016). Importantly, PROMs collected through different approaches are valid (Bernstein et al., 2020). When it comes to collecting a DMH, prior research has shown that patient self-reported health information is accurate and contains important health history information pertinent to care (Boissonnault & Badke, 2005). Thus, this work builds upon this important knowledge with novel insight by demonstrating what approach to collecting DMHs leads to the highest response rate, considering the resource utilisation required.

As the COVID-19 global pandemic progresses and slowly resolves, it is likely that some of the progress made in routinely incorporating telehealth and at-home monitoring and data collection, including obtaining a DMH, will continue. In practice, our findings suggest that DMHs can be collected prior to musculoskeletal clinic visits, though not at a definitely high rate. While one may expect telephone calls to increase completion rates, we found this was not necessarily the case. Additionally, such an approach requires a great deal of resources, including personnel and time; thus, it may be more plausible and cost-effective to utilise an email approach to collect DMHs. This is because it is less resource intensive. If this approach is taken, our results suggest utilising a personalised email may lead to a higher rate of DMH completion prior to clinic visits. As we move beyond the need to utilise DMHs as a requirement at our institution to seek musculoskeletal care during the COVID-19 global pandemic and seek to implement it as part of routine, efficient care access, future research will be required to assess if certain patient subgroups (e.g., atraumatic vs. traumatic conditions) and their surgeons benefit from acquiring a DMH prior to clinic visits more than others. Further, greater cost-effectiveness work is needed to determine the best trade-off between DMH completion rate and resource utilisation. For example, it may be that contacting patient via a telephone call does not improve the response rate enough to make the time-consuming nature of this method financially feasible and of value. Ultimately, the challenges and opportunities surrounding DMHs are similar to that which was experienced with the initial implementation of electronic medical records (EMRs) (Alexander, 2007); however, we believe there is potential for real positive, lasting change from routine use of DMHs.

If DMHs are to be implemented in a more routine fashion moving forward, we must work to tackle the roadblocks we observed in the present study. Specifically, further efforts and resources may be warranted to ensure the elderly are able to complete DMHs accurately and successfully. It is possible that for elderly individuals, phone calls—while time consuming and resource intensive—may be optimal. However, it may also be that the elderly in a few years will be more technologically savvy and easily able to complete DMHs online with limited issue. Further, patients with concerns regarding their back or non-specific/other anatomic locations also had decreased odds of completing the DMH. This may be secondary to these anatomic locations being related to more chronic or lingering ailments. This would decrease the stress to see an orthopaedic surgeon; therefore, the added task of completing a DMH may not be ‘worth it’ to such individuals. Overall, however, this preliminary study demonstrates the potential of such an approach to clinical data gathering that previously has been under—or not utilised.

5 CONCLUSION

In conclusion, the COVID-19 global pandemic provided a natural experiment setting to examine, via a rapid cycle, semi-randomised A/B testing approach, the success of collecting DMHs. Our findings suggest differences in DMH completion rates based on contact approach (higher for email encounters compared to telephone encounters) and wording (higher for personalised emails compared to generic emails). In times of crisis, health care transformation may occur at a faster rate, likely driven by necessity. DMHs can be valuable in helping triage patients in resource-strapped settings or times of crisis (e.g. COVID-19 global pandemic), but it is also likely that such utility can also be appreciated in non-crisis times as we transition towards value-based health care delivery systems.

ACKNOWLEDGEMENTS

The research received no specific grant funding.

CONFLICT OF INTEREST

All authors report no conflict of interest related to the submitted work.

AUTHOR CONTRIBUTIONS

Rudolf W. Poolman and Diederik H.R. Kempen participated in the design of the quality improvement protocol. DMH was directly involved in protocol implementation. Irina Meijers, Anne Portengen, Amanda Klaassen and Vanessa A.B. Scholtes were also instrumental in the implementation of the DMH quality improvement project and data management. David N. Bernstein organized and cleaned all data. David N. Bernstein performed the statistical analysis. All the authors participated in interpreting the results and writing of the manuscript. All the authors read and approved the final manuscript.

ETHICS STATEMENT

The present rapid cycle, semi-randomised A/B testing approach observational study was exempt from institutional review board (IRB) approval because this work is an ongoing quality improvement project in the context of routine clinical care. Further, we assessed DMH response rates and did not utilize identifiable patient health data/protected health information (PHI). The study received no specific grant funding. The authors declare no related conflicts of interest.

Research data are not shared.

REFERENCES

Alexander, I. (2007). Electronic medical records for the orthopaedic practice. Clinical Orthopaedics and Related Research, 457, 114– 119. https://doi.org/10.1097/BLO.0b013e3180342802 Bernstein, D. N., Baumhauer, J. F., Kang, J. D., & Makhni, M. C. (2020). Transforming the orthopaedic patient experience through telemedicine. Journal of Patient Experience, 7(3), 302– 304. https://doi.org/10.1177/2374373520929449 Bernstein, D. N., McIntyre, A. W., & Baumhauer, J. F. (2020). Effect of assessment administration method and timing on patient-reported outcome measures completion and scores: Overview and recommendations. Musculoskeletal Care, 18(4), 535– 540. https://doi.org/10.1002/msc.1476 Bini, S. A., Schilling, P. L., Patel, S. P., Kalore, N. V., Ast, M. P., Maratt, J. D., Lawrie, C. M., Chung, C. C., & Steele, G. D. (2020). Digital orthopaedics: A glimpse into the future in the midst of a pandemic. The Journal of Arthroplasty, 35(7s), S68– s73. https://doi.org/10.1016/j.arth.2020.04.048 Boissonnault, W. G., & Badke, M. B. (2005). Collecting health history information: The accuracy of a patient self-administered questionnaire in an orthopedic outpatient setting. Physical Therapy, 85(6), 531– 543. Crawford, A. M., Lightsey, H. M., Xiong, G. X., Striano, B. M., Greene, N., Schoenfeld, A. J., & Simpson, A. K. (2021a). Interventional procedure plans generated by telemedicine visits in spine patients are rarely changed after in-person evaluation. Regional Anesthesia & Pain Medicine. https://doi.org/10.1136/rapm-2021-102630 Crawford, A. M., Lightsey, H. M., Xiong, G. X., Striano, B. M., Schoenfeld, A. J., & Simpson, A. K. (2021b). Telemedicine visits generate accurate surgical plans across orthopaedic subspecialties. Archives of Orthopaedic and Trauma Surgery, 1– 8. https://doi.org/10.1007/s00402-021-03903-2 Horwitz, L. I., Kuznetsova, M., & Jones, S. A. (2019). Creating a learning health system through rapid-cycle, randomized testing. New England Journal of Medicine, 381(12), 1175– 1179. https://doi.org/10.1056/NEJMsb1900856 Iacobucci, G. (2020). Covid-19: All non-urgent elective surgery is suspended for at least three months in England. BMJ, 368, m1106. https://doi.org/10.1136/bmj.m1106 Jain, A., Jain, P., & Aggarwal, S. (2020). SARS-CoV-2 impact on elective orthopaedic surgery: Implications for post-pandemic recovery. The Journal of bone and joint surgery. American volume. 102(13), e68. https://journals.lww.com/jbjsjournal/Fulltext/9000/SARS_CoV_2_Impact_on_Elective_Orthopaedic_Surgery_.99722.aspx Lightsey, H. M. t., Crawford, A. M., Xiong, G. X., Schoenfeld, A. J., & Simpson, A. K. (2021). Surgical plans generated from telemedicine visits are rarely changed after in-person evaluation in spine patients. The Spine Journal, 21(3), 359– 365. https://doi.org/10.1016/j.spinee.2020.11.009 Mehrotra, A., Chernew, M., Linetsky, D., Hatch, H., & Cutler, D. (2020). The impact of the COVID-19 pandemic on outpatient visits: A rebound emerges. https://www.commonwealthfund.org/publications/2020/apr/impact-covid-19-outpatient-visits Palmen, L. N., Schrier, J. C., Scholten, R., Jansen, J. H., & Koëter, S. (2016). Is it too early to move to full electronic PROM data collection? A randomized controlled trial comparing PROM's after hallux valgus captured by e-mail, traditional mail and telephone. Foot and Ankle Surgery, 22(1), 46– 49. https://doi.org/10.1016/j.fas.2015.05.001 Rizzi, A. M., Polachek, W. S., Dulas, M., Strelzow, J. A., & Hynes, K. K. (2020). The new 'normal': Rapid adoption of telemedicine in orthopaedics during the COVID-19 pandemic. Injury, 51(12), 2816– 2821. https://doi.org/10.1016/j.injury.2020.09.009 Schwartzenberger, J., Presson, A., Lyle, A., O'Farrell, A., & Tyser, A. R. (2017). Remote collection of patient-reported outcomes following outpatient hand surgery: A randomized trial of telephone, mail, and E-mail. Journal of Hand Surgery (American Volume), 42(9), 693– 699. https://doi.org/10.1016/j.jhsa.2017.05.002 Thomas, H., & Lee, M. (2020). The invisible patient: Caring for those without COVID-19. https://catalyst.nejm.org/doi/full/10.1056/CAT.20.0139