For hospitalized patients, discharge planning involves identifying the resources and ongoing care a patient will need once they transition out of the hospital and working to make sure these needs will be met.1 Effective discharge planning is critical for ensuring patient safety, limiting medical complications, reducing hospital readmissions, and containing health care costs.

Physical therapists (PTs) working in an acute care setting have a wide range of roles and responsibilities.2 One of their key roles is to assist with discharge planning by making recommendations regarding a patient’s optimal discharge location and ongoing rehabilitation needs.2-5 In fact, effective discharge planning is considered a core competency for entry-level acute care physical therapy practice.6 PTs typically base their discharge recommendations on a variety of factors, including the patient’s preferences, medical status, level of cognitive and physical functioning, environmental factors, available support at home, etc.4,5 While PTs appear to be adept at making discharge recommendations,5 it is still very challenging, as there are multiple factors to consider, and patient needs vary widely.

REVIEW OF LITERATURE

Experts in human judgment and decision-making have noted that there is often inconsistency in decision-making when people are faced with complexity.7 Considering the complex/multifactorial nature of discharge recommendations, we expect there to be inconsistencies among different acute care PTs. For example, while one PT may recommend a patient be discharged home, another may recommend discharge to a skilled nursing facility or to inpatient rehabilitation.

Prior studies have been conducted to examine acute care discharge planning among therapists. For example, Jette et al.4 conducted a study to describe the discharge planning process and understand factors therapists consider when making discharge recommendations. They report that therapists often rely on their knowledge gained from formal education and experience when making discharge recommendations. Other studies have examined how patient outcomes vary when PTs’ discharge recommendations are followed vs. when they are not.5,8 These studies have generally found that patient outcomes are better when PTs’ discharge recommendations are implemented. However, we are not aware of a previous study examining the level of consistency of discharge recommendations made by acute care PTs, when provided with a consistent set of patient cases.

Clinical vignettes can serve as useful tools for examining clinical decision-making.9,10 A clinical vignette is a hypothetical patient case that provides clinicians with pertinent information needed to make a clinical decision (such as a discharge recommendation). Clinical vignettes have been used to examine consistency among clinicians when making a wide range of clinical decisions related to diagnosis, selecting appropriate treatment options, and determining the need for referral.11-16 We believe this approach could be useful for examining the level of consistency in discharge recommendations made by PTs working in acute care.

The purpose of this study was to examine the consistency (inter-rater reliability) of discharge recommendations made by PTs with experience working in an acute care setting, when provided with a consistent set of clinical vignettes describing hospitalized patients.

SUBJECTS

A convenience sample of PTs was recruited from three different level-II trauma centers: Lutheran Hospital, Parkview Regional Medical Center, and Parkview Hospital Randallia. All data associated with this study were collected using an electronic survey, which was distributed in September 2023. Information about this study (e.g., purpose and expectations) and a link to the survey were shared via email with 78 acute care PTs from across the three hospital rehabilitation departments. The email lists used for recruitment and survey dissemination were provided by the rehabilitation department directors. The introduction page of the survey included a statement indicating to the PTs that survey submission reflected consent to having their responses used for the purpose of this study. The information page also indicated that only the investigators directly involved with the study would be able to see the survey responses, to maintain confidentiality. No incentives were provided to participants. This study protocol was approved by the Institutional Review Board at Trine University.

METHODS

As part of this pilot study, survey respondents were provided with 10 different clinical vignettes. The vignettes incorporated different body systems (e.g., musculoskeletal, cardiopulmonary, and neurological), medical conditions, and levels of complexity. Each vignette included a description of a patient’s medical condition, past medical history, prior level of function, living situation, key examination findings, and other general characteristics (e.g., age). The vignettes were developed by four student PTs with acute care clinical experience and a supervising PT with a degree in physical therapy and 26 years of acute care physical therapy experience. Vignettes were modeled on patients routinely seen in an acute care hospital. An example vignette is included as an appendix (Appendix 1) and the complete list is provided as supplemental material: https://links.lww.com/JACPT/A24.

After reviewing each case, PT respondents were asked to select a discharge recommendation from the following list of options: home health, inpatient rehabilitation, long-term acute care, outpatient physical therapy, skilled nursing, subacute rehabilitation, no physical therapy needs at discharge, or other. These options were based on the discharge recommendations routinely made by physical therapists.17 We did not define care settings since we did not want to influence the PTs’ recommendations by providing prompts about the level/type of care provided in different settings. Instead, we wanted recommendations to be based on each PT’s personal understanding of the unique aspects of each setting. When therapists selected “other,” they were asked to explain their selection via text response. Responses were recorded using an electronic survey created using Typeform (Barcelona, Spain). Respondents were instructed to complete the survey independently and to not discuss the vignettes with another PT. The survey was available for a 2-week period, with no reminders after the initial email was distributed. We estimated the survey would take 10–15 minutes to complete (the median response time was 13 minutes). Vignettes were on separate pages within the survey. Participants were able to go back and change previous responses but could not save their responses and complete later. Respondents were required to submit their name as part of the survey to limit multiple submissions from the same person.

Statistical Analysis

Fleiss’ kappa was examined to assess the level of agreement (i.e., consistency) among the PTs regarding their discharge recommendations.18 Fleiss’ kappa can be used to capture the level of agreement among multiple raters, beyond what would be expected by chance alone. Fleiss’ kappa values were evaluated according to the following guidelines19: <0.20 = “poor agreement,” 0.21–0.40 = “fair agreement,” 0.41–0.60 = “moderate agreement,” 0.61–0.80 = “good agreement,” 0.81–1.00 = “very good agreement.” A 95% confidence interval (95% CI) was also generated for Fleiss’ kappa. As a follow-up analysis, the kappa statistics associated with each nominal category were also examined to get a sense of how agreement varied across the different response options. We also calculated the percent agreement for each case by expressing the proportion of agreement for each case as a percentage. SPSS software was used for statistical analysis (IBM SPSS Statistics for Windows; Armonk, NY, USA).

RESULTS

Nineteen PTs (13 women and 6 men) completed the survey (19 of 78 possible respondents = 24.4% response rate). Eleven of these PTs had earned a doctorate degree in physical therapy (one transitional doctorate), three had earned a master’s degree, and five had earned a bachelor’s degree. On average (mean ± standard deviation), the PTs reported practicing in an acute care setting for 14.6 ± 11.6 years (ranging from 1 to 36 years). Seventeen of the PTs reported that they had practiced in a setting besides acute care at some point (12 inpatient rehabilitation, 11 outpatient, 11 skilled nursing, and 2 home health).

There were inconsistencies in the PTs’ recommendations across all 10 clinical vignettes (Table 1). Overall, Fleiss’ kappa was 0.28 (95% CI = [0.25, 0.30]), which reflects fair agreement among the PTs regarding their discharge recommendations (Table 2).

TABLE 1. - Physical Therapist Responses for All 10 Clinical Vignettes Case Home Health Inpatient Rehab LTAC Outpatient PT Skilled Nursing Subacute Rehab Other Percent Agreement (%) 1 3 1 0 15 0 0 0 63.2 2 0 4 0 0 12 3 0 43.9 3 0 18 0 0 1 0 0 89.5 4 0 5 1 0 7 6 0 26.9 5 2 14 1 0 0 1 1 53.8 6 0 14 0 1 2 2 0 54.4 7 0 14 0 0 3 2 0 55.6 8 5 2 1 1 6 3 1 17.0 9 9 6 0 1 1 1 1 29.8 10 0 5 0 0 11 2 1 38.6

Case 1–10 represents each clinical vignette.

Numbers in the cells represent the number of physical therapists who selected each option.

% agreement = percent agreement among physical therapists.

Rehab = rehabilitation; LTAC = long-term acute care; PT = physical therapy.

The response option “no physical therapy needs at discharge” is omitted from this table, since it was never selected.

TABLE 2. - Level of Agreement Overall and for Each Response Option Fleiss’ Kappa 95% Confidence Interval Level of Agreement Overall agreement 0.28 0.25, 0.30 Fair Home health 0.21 0.17, 0.26 Fair Inpatient rehabilitation 0.34 0.29, 0.38 Fair Long-term acute care hospital −0.02 −0.06, 0.03 Poor Outpatient physical therapy 0.61 0.56, 0.66 Good Skilled nursing 0.25 0.20, 0.29 Fair Subacute rehabilitation 0.03 −0.02, 0.08 Poor Other −0.02 −0.07, 0.03 Poor

Fleiss’ kappa values and 95% confidence intervals (lower bound and upper bound) overall and for each response option.

The PTs exhibited moderate-to-good agreement when recommending outpatient physical therapy (kappa = 0.61); fair agreement when recommending home health (kappa = 0.21), inpatient rehabilitation (kappa = 0.34), and skilled nursing (kappa = 0.25); and poor agreement for all other response options (kappa ≤ 0.03) (Table 2).

DISCUSSION AND CONCLUSION

The purpose of this pilot study was to examine the consistency of discharge recommendations made by PTs with experience working in an acute care setting. Overall, the PTs in our study exhibited only fair agreement with respect to their discharge recommendations (overall kappa coefficient = 0.28). While this relatively low level of agreement is not necessarily surprising given the complex nature of making decisions regarding a patient’s optimal discharge location/needs, it is noteworthy and suggests that efforts to improve the consistency of discharge recommendations should be explored. This study expands upon the body of literature related to discharge planning, supplementing previous studies that have described the discharge planning process,4 identified factors therapists consider when making discharge recommendations,4 and explored how patient outcomes vary when PTs’ discharge recommendations are followed vs. when they are not.5,8

The level of consistency varied considerably among the clinical vignettes (although there was not perfect agreement for any case). For some vignettes, the level of agreement was quite poor. For instance, case 4 (appendix) described a 64-year-old male with chronic obstructive pulmonary disease (along with multiple comorbidities) recovering after a bullectomy. For this case, seven PTs (37%) recommended skilled nursing, six (32%) recommended subacute rehabilitation, five (26%) recommended inpatient rehabilitation, and one (5%) recommended long-term acute care. While the PTs all agreed that the patient required ongoing medical supervision/care, the level varied considerably. In other cases, the level of inconsistency was even more striking, with recommendations ranging from outpatient physical therapy to subacute care (cases 6, 8, and 9). These cases highlight how challenging it can be to make discharge recommendations, even for PTs with acute care experience. This level of inconsistency is also somewhat concerning, as it suggests that recommendations could be at a level that is above or below what is necessary for certain patients. However, it is worth noting that it is currently unclear what an acceptable level of consistency is when it comes to discharge recommendations.

In general, more complex cases tended to produce greater levels of inconsistency. For example, case 1, which was a relatively simple case of a patient who had undergone total knee arthroplasty, exhibited much greater agreement than case 8, which described a more complex case where the patient was experiencing a psychiatric disorder following ankle surgery (Table 1: case 1 = 63.2% agreement vs. case 8 = 17.0% agreement). This suggests that not all patient cases present the same level of challenge to clinical decision-making (i.e., some cases are more “straight-forward” than others).

A next step in this line of research is to explore ways to reduce inconsistencies among PTs. One way may be to create opportunities for PTs to discuss sample cases together and with other members of the discharge planning team. Reviewing and discussing cases may help PTs better understand how their colleagues weigh different factors when making discharge recommendations and would provide an opportunity to clarify the differences between the level/type of care in different settings (e.g., inpatient rehabilitation vs. long-term acute care vs. skilled nursing vs. subacute rehabilitation). It may also be beneficial for PTs to collaborate when making discharge recommendations so different perspectives are considered, especially for complex patient cases.

There may also be opportunities to leverage predictive models to assist with making discharge recommendations, as this has been shown to limit inconsistent decision-making in other contexts.20 Predictive models of this nature are used regularly to support complex clinical decision-making in other areas of medicine. For example, the CURB-65 Pneumonia Severity Assessment is a predictive model used as a guide for determining whether a patient with community-acquired pneumonia should be hospitalized or not.21 The CURB-65 incorporates a patient’s cognitive state, blood urea nitrogen level, respiratory rate, blood pressure, and age to help determine the appropriate clinical course of action (inpatient vs. outpatient treatment). This same concept could be applied to develop a predictive model to guide PTs’ discharge recommendations. Standardized assessment tools, such as the AM-PAC “6-Clicks,”22,23 are already available to quantify a patient’s level of mobility and independence. Predictive models could incorporate patient scores on these types of standardized assessments, along with other pertinent factors (e.g., age, cognitive status, and home support). Regardless, it is important to note that these predictive models would not be a replacement for PTs’ clinical judgment. Instead, they would serve as a form of clinical decision support,24 leveraging existing patient data and information technology to inform discharge recommendations.

The level of inconsistency among PTs suggests there are opportunities to improve discharge recommendations for hospitalized patients. That said, it should be noted that there is evidence to suggest that PTs are adept at making discharge recommendations. For example, Smith et al.5 found that patients who were discharged to the location recommended by a PT were less likely to be readmitted to the hospital. Wright et al.8 also found that patients discharged to the location recommended by a PT were less likely to experience a fall after hospital discharge. Again, while the lack of consistency should be viewed as an opportunity for improvement, previous research suggests that PTs are in a good position to make discharge recommendations for hospitalized patients.

While this study provides useful insights regarding the consistency of PTs’ discharge recommendations, there are limitations that should be considered. First, the use of clinical vignettes could be viewed as a limitation, as it is unclear how closely these types of hypothetical patient cases reflect “real-world” clinical situations.9,10 However, there is evidence to suggest that responses to clinical vignettes tend to align with health care providers’ actual behaviors, suggesting that clinical vignettes can be used to gain insights regarding clinical decision-making.10 Second, the PTs who participated in this study varied considerably with respect to their education levels and years of clinical experience. In the future, we plan to examine how responses vary among PTs with different levels of training and experience. This would require a larger sample of PTs than what was included in this pilot study. It is also important to note that the consistency among PTs would have been better if we had opted for fewer response categories; however, we wanted to provide all the options PTs would likely consider when making a discharge recommendation to make this study more authentic to clinical practice. Finally, the vignettes were not reviewed by experts, beyond those directly involved with this study.

Although not a limitation, it is also worth noting that the vignettes represented a patient’s status at the point of initial evaluation since PTs are typically asked to make a recommendation immediately post-evaluation to allow time for discharge planning. However, a patient’s status often changes throughout a hospital stay, and thus, discharge recommendations can change as well. This highlights the need for PTs to have strong prognostication skills, as well as the importance of developing valid prognostic tools, to help project a patient’s recovery.25

In conclusion, PTs with acute care experience exhibited only fair agreement when making discharge recommendations based on clinical vignettes describing hospitalized patients. Future studies should examine factors that contribute to these inconsistencies, as well as strategies for promoting more consistency among PTs.

ACKNOWLEDGMENTS

The authors would like to thank the physical therapists who volunteered to participate in this study.

REFERENCES 1. Alper E, O’Malley TA, Greenwald J. Hospital discharge and readmission. UpToDate; 2023. Accessed October 29, 2023. https://www.uptodate.com/contents/hospital-discharge-and-readmission. 2. Masley PM, Havrilko C-L, Mahnensmith MR, Aubert M, Jette DU. Physical therapist practice in the acute care setting: a qualitative study. Phys Ther. 2011;91(6):906-919. 3. Falvey JR, Burke RE, Ridgeway KJ, Malone DJ, Forster JE, Stevens-Lapsley JE. Involvement of acute care physical therapists in care transitions for older adults following acute hospitalization: a cross-sectional national survey. J Geriatr Phys Ther. 2019;42(3):E73–E80. 4. Jette DU, Grover L, Keck CP. A qualitative study of clinical decision making in recommending discharge placement from the acute care setting. Phys Ther. 2003;83(3):224-236. 5. Smith BA, Fields CJ, Fernandez N. Physical therapists made accurate and appropriate discharge recommendations for patients who are acutely ill. Phys Ther. 2010;90(5):693-703. 6. Greenwood KC, Stewart E, Hake M, Milton E, Mitchell L, Sanders B. Defining entry-level practice in acute care physical therapist practice. J Acute Care Phys Ther. 2017;8(1):3-10. 7. Kahneman D, Sibony O, Sunstein CR. Noise: A Flaw in Human Judgement. 1st ed. Little, Brown Spark; 2021. 8. Wright JR, Koch-Hanes T, Cortney C, Lutjens K, Raines K, Shan G, Young D. Planning for safe hospital discharge by identifying patients likely to fall after discharge. Phys Ther. 2022;102(2):pzab264. 9. Converse L, Barrett K, Rich E, Reschovsky J. Methods of observing variations in physicians’ decisions: the opportunity for clinical vignettes. J Gen Intern Med. 2015;30(S3):S586–S594. 10. Evans SC, Roberts MC, Keeley JW, et al. Vignette methodologies for studying clinicians’ decision-making: validity, utility, and application in ICD-11 field studies. Int J Clin Health Psychol. 2015;15(2):160-170. 11. Abady AH, Rosedale R, Overend TJ, Chesworth BM, Rotondi MA. Inter-examiner reliability of diplomats in the mechanical diagnosis and therapy system in assessing patients with shoulder pain. J Man Manip Ther. 2014;22(4):199-205. 12. Dale PC, Thomas JC, Hazle CR. Physical therapist clinical reasoning and classification inconsistencies in headache disorders: a United States survey. J Man Manip Ther. 2020;28(1):28-40. 13. Ikezawa Y, Battie MC, Beach J, Gross D. Do clinicians working within the same context make consistent return-to-work recommendations? J Occup Rehabil. 2010;20(3):367-377. 14. Maj M, Pirozzi R., Formicola AM, Bartoli L, Bucci P. Reliability and validity of the DSM-IV diagnostic category of schizoaffective disorder: preliminary data. J Affect Disord. 2000;57(1-3):95-98. 15. Nuttall J, Evaniew N, Thornley P, et al. The inter-rater reliability of the diagnosis of surgical site infection in the context of a clinical trial. Bone Joint Res. 2016;5(8):347-352. 16. Ploumis A, Phan P, Hess K, Wood KB. Factors influencing surgical decision making in adult spine deformity: a cross-sectional survey. Spine Deform. 2014;2(1):55-60. 17. Jette DU, Brown R, Collette N, Friant W, Graves L. Physical therapists’ management of patients in the acute care setting: an observational study. Phys Ther. 2009;89(11):1158-1181. 18. Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76(5):378-382. 19. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-174. 20. Kahneman D, Rosenfield AM, Gandhi L, Blaser T. Noise: how to overcome the high, hidden cost of inconsistent decision making. Harvard Business Review. 2016;94(10):38-46. 21. Jones BE, Jones J, Bewick T, et al. CURB-65 pneumonia severity assessment adapted for electronic decision support. Chest. 2011;140(1):156-163. 22. Arnold SM, Naessens JM, McVeigh K, White LJ, Atchison JW, Tompkins J. Can AM-PAC “6-Clicks” inpatient functional assessment scores strengthen hospital 30-day readmission prevention strategies? Cureus. 2021;13(5):e14994. 23. Jette DU, Stilphen M, Ranganathan VK, Passek SD, Frost FS, Jette AM. AM-PAC “6-Clicks” functional assessment scores predict acute care hospital discharge destination. Phys Ther. 2014;94(9):1252-1261. 24. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3(1):17. 25. Tousignant-Laflamme Y, Houle C, Cook C, Naye F, LeBlanc A, Decary S. Mastering prognostic tools: an opportunity to enhance personalized care and to optimize clinical outcomes in physical therapy. Phys Ther. 2022;102(5):pzac023. Appendix. 1 - Example of a clinical vignette (#4 out of 10)

Background information:

64-year-old male with chronic obstructive pulmonary disease (COPD)

Postoperative day number: 1

Current status: patient underwent bullectomy due to exacerbation of COPD symptoms. Refused therapy postoperative day 0 due to increased pain. He is hesitant to attempt physical therapy but agrees to try. Patient’s oxygen levels between 88-91% throughout session.

Patient’s discharge plan: patient wants to return home as he is worried about his dogs; however, he is nervous about being able to care for himself. Patient’s son is adamant that patient returns home as soon as possible. Son insists he will fix handrail and stay in the spare bedroom on the first floor of the duplex. Patient is hesitant regarding plan because son stays out until early in the mornings hanging out with high school friends and sometimes does not come home for days. Patient is willing to go home to see his dogs.

Patient history:

Past medical history: hypertension, cellulitis, atrial fibrillation, lymphedema, obesity, pulmonary embolism, osteoarthritis

Medications: benazepril, dicloxacillin, ketoprofen

Prior level of function: lived alone. Cared for his 2 dogs. Ambulated independently with rolling walker. Does not drive. Does laundry and other household chores when absolutely necessary.

Hobbies: watching the Price is Right and watching his dogs run around the backyard.

Pain rating: 4/10 at rest, 6/10 with movement

Home setup: lives on 1st floor of duplex. 3 steps to enter; broken left handrail. Bedroom and bathroom on 1st floor. Son lives on 2nd floor of duplex.

Durable medical equipment: rolling walker

Examination findings:

Cognition: alert and oriented x 4

Active range of motion, bilateral upper extremities: within normal limits, except shoulder flexion/abduction limited to 90ᴼ

Active range of motion, bilateral lower extremities: within normal limits, except knee flexion = 100ᴼ, knee extension = −15ᴼ

Manual muscle testing: bilateral lower extremities = grossly 3+/5; bilateral upper extremities = 4-/5

Sensation: light touch impaired, bilateral lower extremities

Bed mobility: moderate assist x 2

Transfers: maximum assist x2 for stand pivot

Locomotion: not attempted

Balance: sitting = poor; heavy posterior trunk lean

Activity Measure for Post-Acute Care (AM-PAC): 6/24