Introduction

With increasing healthcare costs and an ageing population, identifying and resolving causes of low-value care is an important issue.1 Low-value care is broadly defined as care that provides little or no additional benefit to patients with respect to the associated risks of harm and healthcare costs.2

It is estimated that 70% of clinical decisions are informed by pathology testing.3 4 This has likely driven the increase in ordering observed over the last two decades, with a rise from 40 million tests in the year 2000 to 90 million in 2015, accounting for 3% of the Australian healthcare budget in 2015.5 Furthermore, pathology testing can trigger a cascade of additional tests, investigations and procedures.6 Underordering of pathology tests is a broad and complex problem which is outside the scope of this study.3 It is vital to optimise efficiency in this area of healthcare to maintain and improve the quality and cost-effectiveness of care.2 5

It is estimated that between 21% and 69% of all laboratory tests are unnecessary.3 7 There is extreme variability in outcomes from these studies because the analyses have ranged from subjective to objective, with both restrictive and permissive criteria.3 Test appropriateness is determined through risk/benefit analysis and only some clinical scenarios have good evidence for appropriate test or retest intervals.8–10 Objective measures, such as minimum retest intervals, are useful tools because they can categorise tests as clearly appropriate or inappropriate regardless of the clinical context.10

Studies have shown patterns of extended, habitual, repeat testing for common laboratory tests that are low yield and clinically unjustified.11 12 Our previous study found that repeat full blood count (FBC) and Urea-electrolyte-creatinine (UEC) tests were ordered every 24 hours for patients with a Length of Stay (LOS) >2 days, even when the previous result was within normal limits.11 Only 0.3% of patients received a high-risk result following a normal UEC test. This suggests that routine pathology testing to detect emerging problems is very low yield.11 Most physicians (90%) cited a training culture that emphasised repeat daily tests when they were asked why they ordered unnecessary tests.13 Clinical guidelines that explicitly address this low-value clinical culture, and lessen the subjectivity and permissiveness of laboratory test ordering, may help reduce low-value care.14

It has been hypothesised that electronic ordering (e-ordering) (given various names in the literature, including electronic requesting, e-requesting or computerised physician order entry) systems could address problems of low-value care by decreasing laboratory test turnaround time and reducing the volume of tests ordered, including reducing duplicate tests.15–17 E-ordering systems are electronic ordering systems for medications and clinical investigations such as pathology and imaging.15 To date, the effect of e-ordering systems on pathology test ordering patterns and patient outcomes has been mixed.18–21 Studies have shown improved outcomes following e-ordering implementation in various settings (emergency department (ED), inpatient, outpatient). These include decreased total volume of tests ordered (ED, outpatient),18 19 22 reduced repeat labs (inpatient)22 and decreased test turnaround time and patient LOS.21 23 24 Others have shown no effect on volume of inpatient test orders18 20 or patient outcomes including intensive care readmission, hospital mortality or ventilator days.25–28 Yet other studies have shown a negative impact of the e-ordering system such as increased test orders (ED)21 and increased medical errors and adverse drug events.15 29–31; this may potentially be facilitated by habitual reordering and creating standing orders.7 13 32 Previous studies have investigated various pathology tests to measure overordering. In this study, only the most commonly ordered, or commonly over-ordered, pathology tests were examined.33 For example, FBC was found to be the most commonly ordered test in our previous study11 and vitamin D has been identified as a commonly over-ordered test by the ‘Choosing Wisely’ campaign.34 35

E-ordering is still relatively new in Australia and few studies have investigated the effect of digital systems on laboratory test ordering patterns in local hospitals.20–22 These studies produced inconsistent results, likely caused by study and technological limitations, for example, short collection periods, inadequate ward involvement and narrow patient populations.20–22 To address these issues, this study aimed to analyse the volume of test ordering before, during and after e-ordering implementation to determine if the new e-ordering system had any impact.

MethodsStudy design

An audit and analysis of pathology test data were conducted, using an interrupted time series design,36 to investigate the impact of the e-ordering system on pathology ordering patterns for inpatients at John Hunter Hospital (JHH), NSW. This paper is a continuation of the Pathology Laboratory Unnecessary Test Ordering study.11 The current study focuses on the impact of the introduction of an e-ordering system on the volume and pattern of test ordering when compared with manual, paper-based, ordering. The e-ordering system implemented does have an alert—a small blue triangle—that indicates when a test has been ordered within the previous 24 hours.

Setting and sample

JHH is the largest tertiary hospital within the Hunter New England Health District. It has onsite laboratory services and approximately 650 beds across medical and surgical wards.11 JHH began the rollout of an e-ordering system in 2018. Pathology test orders were divided into three time periods: 1 January to 31 December 2017 was classified as the ‘pre’ implementation period, 1 January to 31 December 2018 as the ‘during-’ implementation period and 1 January to 31 December 2019 comprised the ‘post’ implementation period. This allowed observation of any pattern changes in laboratory test ordering from preimplementation to postimplementation of the e-ordering system, including the during-implementation period where e-ordering was rolled-out in a phased introduction across all wards over 12 months. Of note, the COVID-19 pandemic had no impact on this study as the data were collected prior to 2020.

Data were collected on all adult (≥18 years of age) medical and surgical patients admitted at JHH from 1 January 2017 to 31 December 2019. Emergency medicine, paediatric, obstetric, gynaecological, intensive care, psychiatric, elective day procedures (eg, endoscopy), rehabilitation, physiotherapy, general practitioner and renal dialysis admissions were excluded as use of pathology (types and frequency of tests) in these wards differs from medical/surgical wards. Contiguous admissions (readmission within 24 hours post-discharge) were collapsed into one admission. Admissions that started or finished outside the audit period were truncated (date of admission assigned as 1 January 2017, date of discharge assigned as 31 December 2019). For each admission, the LOS was rounded up to whole days. The number of occupied hospital ‘bed days’ for each month was determined per patient and summed across patients.

Data collection

Deidentified data were extracted from the Patient Administration System and were supplied by New South Wales Health Pathology (NSWHP) using the Health Information Exchange. This included relevant patient demographics; gender and age for each patient, admission and discharge dates/times (from which LOS was calculated), diagnoses (as per the International Classification of Disease, edition 10 codes) and Charlson Comorbidity Index (CCI). (For specific demographic information related to the assignment of CCI, see online supplemental appendix table A1). Laboratory data, including pathology tests ordered and results, were extracted from Auslab, the laboratory information system used in the Local Health District. Pathology tests in this hospital are ordered using CAP-Orion software.

Seven types of pathology tests were included in this study: UEC, FBC, troponin, thyroid-stimulating hormone (TSH), 25-hydroxyvitamin D (vitamin D), glycated haemoglobin (HbA1c) and C-reactive protein (CRP). Test results were classified as normal, moderate risk and high risk, based on the corresponding reference ranges used by AUSLAB.11

Data analysis

Statistical analysis was undertaken by an independent statistician. Descriptive statistics were presented as count (%) for dichotomous variables or mean and SD (or median (min, max)) if continuous. χ2 test was used for comparison of categorical characteristics among periods. Analysis of variance and Kruskal-Wallis were used for comparison of parametric and non-parametric distributions of continuous data, respectively.

The monthly change over time in aggregated test orders (overall, and by test type) was compared among groups using segmented negative binomial regression modelling (count outcome). Crude modelling included group (preimplementation, during implementation and postimplementation) and three segmented time intervals (pretime, during time, post-time) in months. Robust SEs were used to account for autocorrelation over time. Adjusted modelling included aggregated bed days (mean-centred), to account for any changes in bed days over time. Assumptions for negative binomial regression modelling were checked and found to be reasonable.

A slope estimate, in the form of an incidence rate ratio for a 1-month change in number of tests ordered, and corresponding 95% CIs were reported for each group for the crude and adjusted models. An overall type 3 p value (interaction between group and time) was calculated to assess overall statistical differences in slopes among the preimplementation, during-implementation and post-implementation groups. The comparison in slope estimates between preimplementation and postimplementation groups and its corresponding p value was also reported. Changes over time were visualised by plotting estimated counts (exponentiated least squares means (LS-means)) from adjusted regression models. Estimated intervention effect was calculated in two forms, level change and slope change, between preimplementation and postimplementation groups. This estimates the change in outcome attributable to the intervention while accounting for the preintervention slope for test ordering,37 see figure 1.

Figure 1

Examples of impact models used in interrupted time series. (a) Estimated intervention effect, level change, (b) slope change; (c) level and slope change; (d) slope change following a lag; (e) temporary estimated intervention effect; (f) temporary slope change leading to an estimated intervention effect. Reproduced under the terms of the Creative Commons Attribution License from Bernal JL, Cummins S, Gasparrini A. Int J Epidemiol. 2017 Feb 1;46(1):348-355.doi: 10.1093/ije/dyw098.

All statistical analyses were programmed using SAS V.9.4 (SAS Institute, Cary, North Carolina). A priori, p<0.05 (two-tailed), was used to indicate statistical significance.

Results

A total of 78 760 distinct admissions (comprising of 54 532 patients) were included in this study. Of these, 62 503 (79.4%) had pathology tests and, in total, 2 386 682 pathology tests were ordered over the 3-year study period.

Cohort demographics

There were no clinically important differences observed in age, gender, diabetic status, CCI or LOS across the preimplementation, during-implementation and postimplementation periods (see table 1).

Table 1

Patient demographics across preimplementation, during-implementation, and postimplementation of e-ordering periods

Pathology test-ordering patterns by period

The pathology test-ordering patterns by period (preimplementation, during-implementation and post implementation) are shown in table 2. There was a slight increase in the number of tests ordered overall across time (total number of tests per admission and per bed day). Test orders also slightly increased for UEC, FBC, troponin and CRP, but not for TSH, vitamin D or HbA1c. Figure 2 visualises this increasing trend of total number of tests ordered during the three periods.

Table 2

Pathology ordering volumes, by number of admissions and aggregated bed days for all tests and by test types

Figure 2

Number of pathology tests ordered over time by period. Total number of pathology tests ordered over three periods. Preimplementation, during implementation and postimplementation periods were represented in blue (months 1–12; January 2017 to December 2017), red (months 13–24; January 2018 to December 2018), and green (months 25–36; January 2019 to December 2019), respectively. The shaded areas represent 95% CIs for each time period.

Pathology test results by period

The pathology test results were individually examined for changes across the three time periods. Each test result was categorised as ‘normal’, ‘moderate risk’ and ‘high risk’. The proportion of moderate and high-risk tests was similar across time periods, except for TSH and HbA1c, where there was a slight increase in the proportion of moderate and high-risk results from preimplementation to postimplementation, see online supplemental appendix table A2.

Overall pathology test ordering

A comparison of test ordering among time groups showed increases in overall test ordering seen in the preimplementation (0.4% increase per month) and postimplementation (0.7% increase per month) periods, representing overall average increases of 242 and 489 tests per month, respectively (table 3). However, this was not found to be statistically significant with no significant slope difference (p=0.108) or estimated intervention effect (step difference) (p=0.645) between these periods. The mean-centred bed days were included as a covariate in all adjusted models as it was statistically associated with the outcome in all models (p value <0.001).

Table 3

Adjusted segmented negative binomial regressions (adjusted for mean centred aggregated bed days)

Individual pathology tests

There were no significant differences seen in the monthly change (slope) between preimplementation and postimplementation periods for FBC (p=0.092) and TSH (p=0.433), despite significant type 3 p values for both FBC (p=0.017) and TSH (p=0.005). Although there was no significant estimated intervention (step change) for FBC (p=0.429), there was a significant intervention effect (step change) for TSH from preimplementation to postimplementation periods (23.1% estimated reduction in tests as of January 2019; p=0.038), see table 3 and figure 3. Figure 3 depicts a large increase in TSH tests ordered between preimplementation and during-implementation periods and progressive decline throughout the during-implementation period. This was unexpected as it deviates from the increasing trend seen in the preimplementation and postimplementation periods, and likely accounts for the significant type 3 p value (p=0.005).

Figure 3

Regression lines of estimated counts for thyroid stimulating hormone (TSH) test orders over the three periods. Regression lines (solid) show the estimated counts (after adjusting for bed days) of TSH orders over time. Square symbol overlay represents the actual counts for each month across the three periods.

A significant difference in the slopes between the preimplementation and postimplementation periods for UEC (p=0.030), vitamin D (p=0.010) and CRP (p=0.044) was observed, see figure 4. These may be false positives, as the respective type-3 p values were non-significant: UEC (p=0.090), vitamin D (p=0.099) and CRP (p=0.154). Unlike UEC and CRP, there was a significant estimated intervention effect for vitamin D (p=0.011) with a significant decrease in number of tests estimated in January 2019 (postimplementation period), compared with extrapolated estimates using preimplementation period slope, see table 3 and figure 4.

Figure 4

Regression lines of estimated counts for orders of urea, electrolytes, creatinine (UEC) (4.1), Vitamin D (4.2) and C reactive protein (4.3) over the three periods. Regression lines (solid) show the estimated counts (after adjusting for bed days) of test orders over time. Square symbol overlay represents the actual counts for each month across the three periods.

There were no significant differences in pretest and post-test orders for troponin and HbA1c, see table 3. The remainder of the regression lines, all non-significant, for other pathology tests is found in online supplemental appendix figure A1.

DiscussionVolume of pathology tests overall

This study detected a monthly increase in test orders over the 3-year period. This likely represents the background increase in test ordering that has been reported in previous literature,3–5 rather than a change attributable to the e-ordering system implemented in this study. The small percentage increase in ordering rates (<1%) found in this study, though statistically insignificant, practically represents a large volume of laboratory tests. This increasing trend represents a resource burden on the healthcare system and is a valuable opportunity to reduce low-value care. E-ordering systems offer interventions like feedback strategies and lockout rules, which may be used to reduce inappropriate test ordering.11

Volume of individual pathology tests

There were clinically meaningful increases in the volume of tests ordered for FBC and UEC tests when comparing preimplementation and postimplementation periods. These two tests comprise the bulk of all tests ordered for inpatients and pose the greatest risk for unnecessary repeat daily orders.11 UEC and FBC should be the subject of order-intervention studies in the future considering their prominence in low-value care.11

Vitamin D and CRP showed decreases in the volume of tests ordered preimplementation to postimplementation. Vitamin D showed a large step change (drop) from the preimplementation period as well as a slope change (decreasing slope) in test order rates during implementation. No other test exhibited this change to ordering patterns during the implementation of the e-ordering system. There was no education campaign targeting vitamin D during this time in our hospital and we speculate that unnecessary vitamin D testing may already be on the decline as it has featured in the ‘Choosing Wisely’ campaign for the last few years.34 35 CRP showed no step change but did show a levelling out in its slope, again indicating that ordering for this test is plateauing. No changes were observed for troponin and HbA1c. These four tests are generally ordered for specific clinical contexts, which means they are unlikely to be habitually reordered within admissions like FBC and UEC.

The findings for TSH were unusual. The during-implementation period showed a sharp reduction in TSH ordering, creating a large step change (drop) from the preimplementation period to the postimplementation period, but the slope continued to increase at virtually an identical rate in both the pre and postperiods. No other test exhibited this change to ordering patterns during the implementation of the e-ordering system. No anomalies or errors in the data or ordering system have been identified to account for this. No interventions aimed at reducing test over-ordering were carried out at JHH during this audit. We speculate that this finding is due to individual clinician behaviour but investigating this further is unfortunately not within the scope of the study.

Strengths and limitations

This study is the first of its kind and was built on a strong theoretical and methodological basis. Potential confounders were avoided where possible, including the effect of COVID-19 on pathology ordering, by using data prior to the pandemic. Seasonal variation is accounted for by taking 12 months of data (one calendar year) per period: preimplementation-, during-implementation and postimplementation. A model adjusted for CCI found a non-significant association with the outcome, indicating no confounding. Conversely, mean-centred bed days was significantly associated with test ordering and, therefore, was accounted for in the adjusted model. There were no pathology ordering interventions implemented at JHH during the study period, which could have confounded results. Additionally, the data collection site was a regional hospital, as opposed to the predominantly metropolitan hospitals studied previously in Australia20–22 and this study, therefore, offers novel data. The methodology and power of the study were strengthened by the nature of the audit as an interrupted time series with long periods of data collection (3 years of data in total). Power was also increased by the equal distribution of data points (in number and seasonality) before, during and after the intervention.36

This study was limited by the number of sites and range of tests that were included. A rural site was initially slated to be involved but was following a year behind the JHH e-ordering rollout. This meant that the COVID-19 pandemic would have confounded the data, leading to its exclusion. Additionally, this study did not compare differences by admitting ward and, therefore, could not account for any potential variation between medical and surgical pathology-ordering patterns. More granular-level data, rather than hospital level trends, could be beneficial in future studies. Importantly, our previous study found no difference in ordering pattern by admitting ward at the same study site.11 There was no explanation found for the unexpected ordering pattern for TSH which limits the interpretation of this result.

Conclusion and future research implications

This study indicates that the introduction of an e-ordering system did not, in and of itself, affect the volume of pathology ordering. The ease with which repeat orders could be placed with the e-ordering system did not lead to increased volumes, but neither did the increased visibility afforded by the system mitigate the pre-existing upward-ordering trend. This means that efforts to tackle low-value ordering will need to intentionally target and intervene on this behaviour. E-ordering systems do not achieve this intrinsically. Further work is needed to test the effectiveness of different educational prompts and lockouts on test-ordering behaviour and, importantly, patient outcomes. Pathology test ordering should also be covered in undergraduate medical training curriculum, specialist training programmes and other educational courses to bridge the gap in medical knowledge and awareness of test limitations.

The results of this study confirm the general secular trend in increasing ordering of pathology tests, with a trend line continuing unchanged throughout the study. By extrapolating preimplementation period data and showing no jump-up in ordering during the implementation period, the data demonstrate that the increasing number of tests ordered were due to background rates previously described.3–5 Several international studies have focused on interventions designed to prevent overordering (eg, displaying the cost of the test to the ordering physician, utilising provider education or applying restrictive changes to the e-ordering).14 25 38 Further research into the long-term effects of these interventions on patterns of pathology ordering are needed as these study periods ended 6 to 12 months after the intervention was implemented. Another interesting future research direction is to investigate whether the increased pathology tests provide value by shortening the length of hospital stay and, therefore, lead to overall financial savings.

Patient outcomes should be a primary focus for future studies, given the ongoing trend of increased pathology test ordering. Blood tests inevitably involve pain and discomfort for patients. Reducing unnecessary tests will both address costly low-value care as well as minimise iatrogenic harm. Additionally, the greater the number of tests ordered, particularly tests with poor pretest probability, the greater the chance of finding spurious results.6 This can lead to further testing and interventions that may be unnecessary and potentially harmful to patients.6 14 Future studies need to investigate what effect this trend of increased pathology testing has on patient outcomes more deeply, especially with respect to new e-ordering systems. Is this culture of increased test ordering justified by improved outcomes for patients or does it merely represent the decreased tolerance for uncertainty that pervades modern medical practice?13