Abstract

Objective: To identify the optimal AUSDRISK threshold score to screen for pre-diabetes and diabetes.

Methods: A total of 406 adult patients not diagnosed with diabetes were screened in General Practices (GP) between May and October 2019. All patients received a point of care (POC) HbA1c test. HbA1c test results were categorised into diabetes (≥6.5% or ≥48 mmol/mol), pre-diabetes (5.7–6.4% or 39–47 mmol/mol), or normal (<5.7% or 39 mmol/mol).

Results: Of these patients, 9 (2%) had undiagnosed diabetes and 60 (15%) had pre-diabetes. A Receiver Operator Characteristic (ROC) curve was constructed to predict the presence of pre-diabetes and diabetes; the area under the ROC curve was 0.72 (95%CI 0.65–0.78) indicating modest predictive ability. The optimal threshold cut point for AUSDRISK score was 17 (sensitivity 76%, specificity 61%, + likelihood ratio (LR) 1.96, - likelihood ratio of 0.39) while the accepted cut point of 12 performed less well (sensitivity 94%, specificity 23%, +LR=1.22 -LR+0.26).

Conclusions: The AUSDRISK tool has the potential to be used as a screening tool for pre-diabetes/diabetes in GP practices. A cut point of ≥17 would potentially identify 75% of all people at risk and three in 10 sent for further testing would be positive for prediabetes or diabetes.

Implications for public health: Routine case-finding in high-risk patients will enable GPs to intervene early and prevent further public health burden from the sequelae of diabetes.

Pre-diabetes is a condition where a person's fasting glucose is elevated but has not reached the diabetes threshold and it is a significant risk factor for the development of type 2 diabetes.1 Pre-diabetes is strongly associated with obesity and screening programs are currently targeting obese individuals using screening tools such as The Australian Type 2 Diabetes Risk Assessment Tool (AUSDRISK).2 Although there are other screening tools around the world,3 the AUSDRISK tool has been designed to suit the cultural differences in the Australian population.2 Currently, persons with scores ≥12 are classified as having a high risk of diabetes.2

Diagnosis of pre-diabetes can be identified in three ways: impaired fasting glucose (5.6–6.9 mmol/l); impaired glucose tolerance (Oral Glucose Tolerance Test [OGTT] result of 7.8 and 11.0 mmol/l); or a glycated haemoglobin (HbA1c) test in the range of 5.7–6.4% (39–47 mmol/mol) as per the American Diabetes Association (ADA) guidelines.1 HbA1c can be performed on non-fasting patients, which is ideal for screening.4 HbA1c in the pre-diabetic range has been shown in other studies to strongly predict diabetes within five years if no health or lifestyle intervention takes place.1 This test can be performed through a formal blood test either using an onsite pathology department or using a Point of Care (POC) HbA1c machine. The results from a POC HbA1c test has been shown to have a correlation coefficient of 99% compared with formal laboratory blood testing.5

The POC HbA1c test provides instant results and therefore the General Practitioner (GP) can act on the results while the patient is still in the waiting room. Patients in the pre-diabetic range should be informed and counselled regarding their diabetes and cardiovascular disease (CVD) health risks.1 Intensive lifestyle changes are encouraged to be part of routine pre-diabetic care as current research shows that it can reduce the incidence of diabetes up to 58%.1, 6 Current health guidelines suggest that those who are diagnosed with pre-diabetes should also have an annual review of glycaemic status.1, 6

Unfortunately, many GPs do not identify pre-diabetic blood results as abnormal, and many patients are not informed that they are at risk of diabetes. Regular screening of at-risk patients is not yet routinely done in busy GPs, despite the provision of a Medicare Benefits Schedule (MBS) reimbursement for an annual HbA1c test for those at risk of pre-diabetes and diabetes.6 GP databases also often lack the information required to identify at-risk patients, since physical measures such as height, weight, body mass index (BMI) and waist circumference may not be easily searchable in the GP clinical database. Support for GPs is required to help achieve better screening, education and prevention for patients at risk of diabetes.7

Although the AUSDRISK was originally developed to identify those at high risk of developing diabetes over a five-year time frame,2 we seek to extend its use to identify those with pre-diabetes, the rationale being that this group – if identified early – may be motivated to engage in lifestyle modification and hence prevent the progression to overt diabetes. Previous work has suggested that the duration of pre-diabetes is long enough to warrant a screening program, but what that program should look like was not discussed.8 The AUSDRISK tool is a self-administered questionnaire that patients can complete while in the waiting room, which may help to identify those who should go on to POC HbA1c testing or more formal tests. Therefore, the aim of our study was to determine the optimal AUSDRISK threshold to screen for pre-diabetes/diabetes in GP practices, using the pragmatic clinical trigger of BMI, i.e. a case-finding approach.

Methods Study Population The study included two routes of screening via four GPs within the Hunter New England Local Health District from 21 May 2019 to 31 October 2019.

Route 1: intentional screening of adult men and women patients on GP lists who were judged to be at high risk based on BMI or fasting glucose at any point in the past, and who were willing to come into the clinic. In reality, very few patients had existing fasting glucose on their health records and the vast majority in this route were screened into the study based on BMI.

Route 2: opportunistic screening of patients already at the GP practice for another appointment.

All patients screened were >18 years of age, had no history of diabetes, had not had an HbA1c test within the past year, and did not have a terminal illness or a severe chronic health condition limiting expected life span to less than one year. The patients were asked if they would like to complete a routine screening questionnaire for diabetes (AUSDRISK). All who agreed were taken to a consulting room where the nurse administered the questionnaire and collected the physical measures. On completion of the questionnaire, all patients were offered an HbA1c test using a POC machine (Alere Afinion AS100, Abbott Diagnostics).

This screening was embedded within a larger program aiming to improve diabetes care throughout the local health district, called the Diabetes Alliance program.7 This program uses a case conference model with endocrine specialists visiting practices to upskill GPs and provide advice on the management of existing patients with diabetes in addition to improved case detection (Hunter New England Local Health District Human Research Ethics Committee, approval 15/04/15/5.02). All participants found to have diabetes were referred back to their GP (and subsequent specialist review if needed) and those found to have an HbA1c result within the pre-diabetes diagnosis range were referred to the NSW Health Get Healthy Information and Coaching Service, a free personalised health promotion lifestyle modification initiative from the NSW Ministry of Health.9 Those with pre-diabetes were also invited to participate in the Zinc in Preventing the Progression of Pre-Diabetes (ZIPPeD) study, which is investigating the potential for zinc supplementation to improve glucose handling in pre-diabetes and was funded by a Translational Research Grant from the NSW Health Department.10 This paper discusses the screening aspects of the study only.

Measurement

Data were collected on age, gender, BMI, AUSDRISK Score, waist circumference and POC HbA1c and the results were recorded in the GP clinical software within the practice database. POC HbA1c was categorised as follows: diabetes (≥6.5% or ≥48 mmol/mol); pre-diabetes (5.7–6.4% or 39–47 mmol/mol); or normal (<5.7% or <39 mmol/mol). BMI was categorised as: normal (18.5 to <25 kg/m2), overweight (25 to <30 kg/m2) and obese (>30 kg/m2).

Statistical analyses

Baseline characteristics were summarised for those who tested within HbA1c ranges considered normal, pre-diabetes and diabetes. Continuous values were summarised using mean and SD, with the difference in means tested using Student's t-test. Categorical and ordinal variables were summarised using frequencies, with the differences tested using a chi-squared test.

Logistic regression models were fitted with diabetes or a combination of pre-diabetes and diabetes as the outcomes, and AUSDRISK score as the predictor, with or without the addition of BMI. We combined pre-diabetes/diabetes as outcomes so as not to skew the diagnostic properties by omitting participants from the calculation of sensitivity and specificity. A Receiver Operating Characteristic (ROC) curve was generated and the area under the ROC (AUROC) curve was measured as an indication of prognostic power. The AUROC represents the probability that a randomly selected case scores higher on the measure in question than a randomly selected control. AUROC values range from 0.5, indicating performance no better than chance, to 1.0 indicating perfect accuracy. Given that models fitted to small samples tend to provide optimistic estimates of the AUROC due to overfitting, bootstrapping was used to generate optimism-adjusted estimates for AUROC.11

The optimum threshold on the ROC curve to use as a cut point for prognostication was calculated using Youden's index, i.e. maximising the sum of sensitivity and specificity. All analyses were performed using STATA 15 or SAS 9.4. Assuming an AUROC of 0.7, power of 80%, p-value of 0.05, and that 20% of those tested would have an HbA1c in the pre-diabetic or diabetic range, we aimed to recruit a minimum of 75 participants to reject the null hypothesis that the AUROC equalled 0.5 (no discriminating power).

Results

The sample included 406 patients (178 males, 228 females) aged between 23 and 85 years; roughly half were identified via each screening route. A BMI value was missing for two patients and an AUSDRISK score was missing for three patients, leaving n=403 in the analysis. Baseline characteristics of the sample are shown in Table 1. The majority of patients in this sample (352/404, 87%) had a BMI above the normal threshold and almost all (391/403, 97%) had an AUSDRISK score of six or higher, indicating an increased risk of type 2 diabetes. This reflects a GP-based population in which case-finding would be clinically indicated, i.e. people who would previously have elevated FBG or who are overweight/obese.

Table 1. Demographic characteristics of the entire group, as well as by pre-diabetes or diabetes status.

Characteristic

Class or Statistic

Total (N=406)

None (n=337)

Diabetes classification (based on hbA1c)

p-value∗∗

Pre diabetes (n=60)

Diabetes (n=9)

Sex

Male

178 (43.8%)

141 (41.8%)

29 (48.3%)

8 (88.9%)

0.015

Female

228 (56.2%)

196 (58.2%)

31 (51.7%)

1 (11.1%)

Body Mass Index (BMI) category

Normal

52 (12.9%)

51 (15.2%)

1 (1.7%)

0.032

Overweight

125 (30.9%)

101 (30.1%)

22 (36.7%)

2 (22.2%)

Obese

227 (56.2%)

183 (54.6%)

37 (61.7%)

7 (77.8%)

AUSDRISK score∗

Low <=5

12 (3.0%)

11 (3.3%)

1 (1.7%)

<0.001

Mod 6–8

15 (3.7%)

14 (4.2%)

1 (1.7%)

Mod 9–11

53 (13.2%)

51 (15.2%)

2 (3.3%)

High 12–15

110 (27.3%)

101 (30.1%)

9 (15.0%)

High 16–19

105 (26.1%)

86 (25.7%)

19 (31.7%)

High 20+

108 (26.8%)

72 (21.5%)

28 (46.7%)

8 (100.0%)

Age in years

mean (SD)

59 (9)

58 (9)

62 (8)

63 (5)

0.010

Body Mass Index (BMI)

mean (SD)

31 (6)

31 (6)

32 (6)

33 (4)

0.213

Notes: ∗ AUSDRISK score categories based on predicting high risk of diabetes.3 ∗∗ For categorical variables, p-value is from Fisher's exact test. For continuous variables, p-value is from ANOVA.

Eight patients who received POC HbA1c tests had undiagnosed diabetes and all of these had an AUSDRISK score of 20 or higher. A further 60 patients were classed as having pre-diabetes, of which all but four had an AUSDRISK score greater than 12.

Logistic regression models were constructed for diabetes, with AUSDRISK score as the predictor, with or without BMI. To ensure that there was no spectrum bias due to the two different routes of screening, we also report analyses adjusted for BMI (the basis of the intentional screening route). The AUROC was very high at 0.89 (95%CI 0.83–0.95) and did not change after adjusting for optimism; this indicates that the AUSDRISK score is a powerful predictor of diabetes (Figure 1 and Table 2). A threshold score of ≥20 essentially had a sensitivity of 100% and specificity of 75%, giving a +LR of 4. The addition of BMI to the AUSDRISK score did not improve the predictive power of the model at all.

Table 2. AUROC for the models predicting diabetes only or pre-diabetes/diabetes combined. Adjustment for BMI shows no change in the predictive ability of the models.

Model outcome

AUROC (95%CI)

1. Diabetes

0.89 (0.83, 0.96)

2. Diabetes (adjusted for BMI)

0.89 (0.82, 0.96)

3. Pre-diabetes/diabetes

0.72 (0.65, 0.78)

4. Pre-diabetes/diabetes (adjusted for BMI)

0.72 (0.65, 0.78)

ROC Curve for Diabetes (AUSDRISK score alone and with BMI).

Logistic regression models for the combined outcome of diabetes and pre-diabetes are shown in Figure 2 and Table 2. The AUROC was only modest at 0.72 (95%CI 0.65–0.78) and this did not change with the adjustment for optimism or the addition of BMI as a covariate. The best threshold AUSDRISK score that maximised sensitivity and specificity was 17 and above; at this cut point the sensitivity was 76%, and specificity 61%, with a positive likelihood ratio of 1.96 and a negative likelihood ratio of 0.39. At the previously suggested AUSDRISK cut point score of 12, the diagnostic characteristics were: sensitivity 94%, specificity 23%, +LR=1.22, -LR=0.26. A complete table of cut point values is given in the Supplementary Material.

ROC Curve for Diabetes and Pre-diabetes (AUSDRISK score alone and with BMI).

A more nuanced approach is also to work out multilevel likelihood ratios for different ranges of the AUSDRISK score, as shown in Table 3. As expected, the likelihood ratios indicate that AUSDRISK scores in the lowest range (0–11) virtually rule out any diabetes and pre-diabetes (LR=0.26) and scores in the highest range (≥20) almost triple the risk of disease (LR=2.5).

Table 3. Multilevel likelihood ratios. Scores between 0 and 16 reduce the odds of disease, while scores of 17 or higher increase the odds of disease.

pre-diabetes and diabetes

absent

present

+LR (95% CI)

score

0–11

76

4

0.26 (0.10, 0.68)

12–16

128

12

0.46 (0.27, 0.79)

17–19

59

16

1.34 (0.82, 2.17)

20+

72

36

2.46 (1.82, 3.34)

total

335

68

Discussion

The AUSDRISK risk score is a powerful prognostic indicator of current diabetes with an AUROC of 0.89. At a threshold of ≥20, this has 100% sensitivity and 75% specificity for diabetes. Given the prevalence of disease in our study, the yield for identifying diabetes or pre-diabetes at this threshold would be 33%, meaning that 2/3 of subsequent testing is normal. A confirmatory test that is inexpensive and reliable may still make this a worthwhile screening program, especially if people can be encouraged to adopt lifestyle changes. The eight patients out of 108 that scored ≥20 in our study had not been suspected of having diabetes and would not have been otherwise identified. The other 28 of 108 falling in the pre-diabetic range would represent 46% of those with pre-diabetes. A larger Australian study of a community cohort found that a cut-off of 20 would only identify 6.4% of the population for further testing,12 which might be acceptable, but this needs a full economic assessment.

In our study, we found 69/406 (17%) of people screened were in the pre-diabetic or diabetic range as judged by their HbA1c results. This is lower than the prevalence of 35% seen at baseline in a randomised controlled trial of high-risk men, likely because in that study participants were selected on the basis of BMI 25–40 and AUSDRISK score ≥12.13

To our knowledge, this is the first study to find an AUSDRISK threshold optimised to identify pre-diabetes as well as diabetes. At a score of 12, Malo et al.12 found sensitivities and specificities of 81% and 58% for identifying impaired fasting glucose, although they excluded diabetes. At the same threshold, we found a sensitivity of 94% and specificity of 23% for the combined endpoint of pre-diabetes/diabetes.

Our results suggest that the AUSDRISK score alone is only a moderate predictor of prediabetes/diabetes, given its AUROC of 0.72, and this did not improve with the addition of BMI. At a cut point of 17, we obtained a sensitivity of 76% and a specificity of 61%. This means that we would capture 75% of patients with a diagnosis of pre-diabetes/ diabetes in screening with this tool alone, without much drop in yield; i.e. Positive Predictive Value (PPV) would be 28%, meaning that roughly three in 10 sent for subsequent testing would be positive. At a lower cut point of 12, the sensitivity increases to 94% and the specificity drops to 23%, but at the cost of a drop in yield to 20%, i.e. two in 10 subsequent tests would be positive. These results are an improvement to those of Lee et al.14 who found that an AUSDRISK score of 15 followed by an HbA1c would pick up just over 50% of those at risk with a yield of 20%.

Approximately one-third of those identified via intentional screening (route 1) and virtually all those identified via opportunistic screening (>95%, route 2) participated in the process. The study confirms the feasibility of the AUSDRISK tool in GPs. The AUSDRISK was administered by a nurse and did require a separate room, given the privacy needed to measure waist circumference. Patients were seen for ∼15 minutes prior to their appointment. The results documented here are in the real-world setting of four typical GP practices, not in a dedicated research clinic, hence the prognostic performance is unlikely to be optimistic.

We believe the likelihood of spectrum bias in our study is low. This is partly due to the fact that the two screening routes are similar to what might happen in the GP setting, and to the fact that adjustment for BMI (mainly the intentional screening route) did not change the results compared to normal weight (the opportunistic screening route).

The question of whether to screen for pre-diabetes/diabetes among asymptomatic patients is still fraught with controversy and randomised controlled trials using different methods and thresholds have come to conflicting conclusions, partly because it is not clear whether there is a benefit to early identification and partly because of the sensitivity to cost estimates.15 Nevertheless, previous work has indicated that the duration of pre-diabetes is sufficiently long to warrant a screening program and the potential to intervene and reduce the burden and sequelae of diabetes is enticing.8

Conclusion

Screening for pre-diabetes and diabetes is feasible in GP practices using the AUSDRISK self-administered questionnaire. Although the prognostic performance (AUROC) is only modest at 0.72, a threshold of 17 would pick up about 75% of cases and give a yield of three in 10 with further testing, while a threshold of 12 would pick up almost 95% of cases and give a yield of two in 10 with further testing. Our team has identified a simple and effective screening program that can be embedded within primary care facilities as part of the patient's routine booked appointment. This has already assisted GPs in commencing early interventions, for example, referring patients to the NSW Get Healthy Information and Coaching Service.9

Acknowledgements

This work is jointly supported by the NSW Ministry of Health Translational Research Grant Scheme (TRGS) for the overall funding of the ZIPPeD study (H18/31636) and HNELHD integrated care funding allocated by the NSW Ministry of Health. The funding body was independent of the research.

Laureate Professor John Attia is the guarantor of this work and, as such, had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Ethics approval

This work has the approval of Hunter New England Human Research Ethics approvals: 15/04/15/5.02, 18/07/18/3.03 and The University of Newcastle Human Research Ethics approval: H-2018-0302-959.

Additional supporting information may be found in the online version of this article:

Supplementary Table 1: Diabetes or pre-diabetes.

Supplementary Table 2: Diabetes.

Filename Description azph13181-sup-0001-S1.docx31.5 KB Supplement Material

Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.

References

1 American Diabetes Association. Pre-Diabetes; 2021 [cited 2021 Oct 28]. Available from: https://www.diabetes.org/diabetes-risk/prediabetes. 2Chen L, Magliano DJ, Balkau B, Colagiuri S, Zimmet PZ, Tonkin AM, et al. AUSDRISK: An Australian Type 2 Diabetes Risk Assessment Tool based on demographic, lifestyle and simple anthropometric measures. Med J Aust. 2010; 192(4): 197– 202. 3Tabák AG, Herder C, Rathmann W, Brunner EJ, Kivimäki M. Prediabetes: A high-risk state for diabetes development. Lancet. 2012; 379(9833): 2279– 90. 4Lim SC, Liying DQ, Toy WC, Wong M, Yeoh LY, Tan C, et al. Adipocytokine zinc alpha2 glycoprotein (ZAG) as a novel urinary biomarker for normo-albuminuric diabetic nephropathy. Diabet Med. 2012; 29(7): 945– 9. 5Weinel LM, Summers MJ, Finnis ME, Poole A, Kar P, Chapman MJ, et al. Are point-of-care measurements of glycated haemoglobin accurate in the critically ill? Aust Crit Care. 2019; 32(6): 465– 70. 6D'Emden MC, Shaw JE, Jones GR, Cheung NW. Guidance concerning the use of glycated haemoglobin (HbA1c) for the diagnosis of diabetes mellitus. Med J Aust. 2015; 203(2): 89– 90. 7Acharya S, Philcox AN, Parsons M, Suthers B, Luu J, Lynch M, et al. Hunter and New England Diabetes Alliance: Innovative and integrated diabetes care delivery in general practice. Aust J Prim Health. 2019; 25(3): 219– 43. 8Bertram MY, Vos T. Quantifying the duration of pre-diabetes. Aust N Z J Public Health. 2010; 34(3): 311– 4. 9O'Hara BJ, Phongsavan P, Rissel C, Hardy LL, Zander A, Greenaway M, et al. Role of general practice in the utilisation of the NSW Get Healthy Information and Coaching Service. Aust J Prim Health. 2015; 21(2): 182– 8. 10Peel R, Hure A, Wiggers J, McEvoy M, Holliday E, Searles A, et al. Zinc in Preventing the Progression of pre-Diabetes (ZIPPeD Study) – study protocol for a randomised placebo-controlled trial in Australia. Trials. 2019; 20(1): 219. 11Harrell FE, Lee KL, Mark DB. Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996; 15(4): 361– 87. 12Afifi M, Almaghrabi OA, Kadasa NM. Ameliorative effect of zinc oxide nanoparticles on antioxidants and sperm characteristics in streptozotocin-induced diabetic rat testes. Biomed Res Int. 2015; 2015: 153573. 13Aguiar EJ, Morgan PJ, Collins CE, Plotnikoff RC, Callister R. Characteristics of men classified at high-risk for type 2 diabetes mellitus using the AUSDRISK screening tool. Diabetes Res Clin Pract. 2015; 108(1): 45– 54. 14Lee CMY, Versace VL, Malo JA, Shaw JE, Dunbar JA, Colagiuri S. Screening for diabetes prevention with diabetes risk scores - A balancing act. Diabetes Res Clin Pract. 2018; 135: 120– 7. 15Simmons D, Zgibor JC. Should we screen for type 2 diabetes among asymptomatic individuals? Yes. Diabetologia. 2017; 60(11): 2148– 52.