To our best knowledge, this study is the first to externally validate and recalibrate existing risk equations for predicting DKD progression in a Taiwanese population with T2D. Although the discrimination of the risk equations was acceptable, the equations underestimated the risks of microalbuminuria and macroalbuminuria and overestimated the risk of renal failure in this population. The risk equations’ performance in calibration was improved after rigorous recalibration procedures. These recalibrated risk equations can be incorporated into a multi-state simulation model to project and differentiate individual patient risks of DKD progression for supporting clinical management and economic research. This study also demonstrates an explicit, structured process for adapting and updating risk equations in different settings and populations.

Comparative discriminative performance of risk equations for DKDs in the Taiwanese population

A discrepancy in model performance regarding discrimination between original development cohorts and external validation cohorts has been reported in previous studies [17]. Specifically, for albuminuria events, only the RECODe model differentiates microalbuminuria and macroalbuminuria events in two separate risk equations. Basu et al. validated the RECODe equation using a longitudinal cohort of African Americans and showed a relatively low discrimination on macroalbuminuria events in these patients compared to that observed in the original development cohort, which predominately comprised Caucasians (i.e., AUROCs: 0.77 versus 0.84) [17]. Similarly, a relatively low but acceptable discrimination of the RECODe equation was found for macroalbuminuria in our Taiwanese population (i.e., AUROC: 0.76). Moreover, the CHIME equation for renal failure yielded better discrimination (AUROC: 0.77) for the Taiwanese population compared to the 2 other equations for renal failure (0.64 and 0.60 from using RECODe and UKPDS-OM2, respectively) [7, 8].

These findings may be partly explained by the differences in patient clinical characteristics and time periods between the original development and external validation cohorts [18]. Specifically, the RECODe and UKPDS-OM2 models were built using data from selective and homogeneous Caucasian patients in clinical trial settings [7, 8] while the CHIME model was constructed using data from a real-world Asian patient cohort (similar to our study patients) [11]. The cohort used for the RECODe model comprised a high proportion of patients with established cardiovascular diseases or associated risk factors, including body mass index, HbA1c, and lipid profiles, which were much higher than those of our study population [7]. Moreover, both the RECODe and UKPDS-OM2 models were developed based on older data (i.e., 2001 − 2009 and 1977 − 1997, respectively) compared to data used for the CHIME model (i.e., 2006 − 2018) [7, 8, 11]. As medical technologies advance, older data might not reflect the modern clinical practice. For instance, the control of comorbid hypertension is considered important in current clinical management for patients with T2D [19], but most hypertensive cases in the UKPDS-OM2 model remained untreated given the few available treatment strategies and associated clinical recommendations at the time when the model was developed [8]. Therefore, owing to the similarity in the composition of the study cohort (i.e., Asians) and study period (i.e., modern era), the CHIME model showed the best discrimination for predicting the renal failure risk in the Taiwanese population (S6 Table).

Recalibration of risk equations to improve predictive performance

The calibration results of the risk equations that indicate the accuracy of estimated risks for albuminuria and renal failure in Taiwanese patients were not satisfactory given a p-value < 0.05 of the GND test. Hence, existing risk equations should not be directly applied to Taiwanese patients without careful model updating and recalibration [20]. Specifically, the risk for albuminuria in Taiwanese patients was underestimated by the RECODe model, as supported by calibration slopes above 1 for both microalbuminuria (i.e., 1.6) and macroalbuminuria (i.e., 4.38). Such underestimation is expected given differences in the study cohort and associated underlying risks of albuminuria. That is, the RECODe model was developed using data from Caucasians primarily enrolled in clinical trials [7], who had a lower risk of albuminuria compared to that of Asians (e.g., Taiwanese) [21]. On the other hand, the risk for renal failure in Taiwanese patients was overestimated by all of the risk equations (i.e., calibration slope < 1), which might be due to the inconsistent definitions of renal failure and the underlying risks of renal failure across studies [22]. For example, renal failure in the CHIME model included dialysis and transplantation, which were ascertained according to associated diagnosis codes [11], whereas our study defined the presence of a renal failure event using two consecutive eGFR records of less than 15 mL/min/1.73 m2 separated by at least 90 days. The lower event rate of renal failure in our study compared to that of the CHIME model (0.13 versus 0.59 per 100 person-years) suggests the potential lower baseline risk of the event in our study cohort. The CHIME model was chosen for further recalibration given its best discriminative performance on renal failure.

To improve the accuracy of the estimated risks of albuminuria and renal failure for the RECODe and CHIME equations, efforts were made to optimize the predictive performance. Given the difference in renal event rates between the development cohort and our cohort, we first adjusted the baseline risk of the overall study cohort using a validated method (S1 Method) for all renal outcomes [14, 16]. However, the recalibrated result for macroalbuminuria remained undesirable following the initial adjustment at a cohort level, as supported by a miscalibration pattern observed in the calibration plot (S5 Fig). In particular, the underestimation was more extreme for high-risk patients (stratum 2) compared to that for low-risk patients (stratum 1), and thus the adjustment for the baseline hazards of the risk equations at a stratum level was further conducted. Our study demonstrated that recalibration of the risk equations improved predictive performances for renal outcomes, suggesting that updating and recalibration of the risk equations are needed to provide more precise estimations of event risks before these equations can be applied to target populations.

Methodological efforts to ensure accuracy and applicability of risk equations

Several efforts were made to enhance the accuracy of estimated risks and applicability of the adapted risk equations for the Taiwanese population. First, sensitivity analyses, in which the definitions of renal outcomes were refined, were conducted. Consistent results across the primary and sensitivity analyses confirm the robustness of the study findings. Moreover, the risk equations were updated using validated methods to improve their prediction accuracy of renal outcomes. Second, a contemporary patient cohort based on EHRs was utilized to reflect a real-world setting with modern clinical practice, with easily obtained and measured clinical characteristics included as the predictors and risk factors. Lastly, we considered DKD progression from the status with normal albuminuria to three DKD subtypes, namely microalbuminuria, macroalbuminuria, and renal failure, which thoroughly depicted the disease progression. Albuminuria is an established risk factor for renal progression and is recognized as an early and sensitive marker for renal damage [23]. However, it is not typically considered in existing risk equations or simulation models.

The updated and recalibrated risk equations obtained in this study could be utilized in healthcare fields. For clinical practice, the risk equations can estimate a patient’s risk of developing a renal event conditional on their clinical characteristics to timely inform effective interventions. Several antidiabetic medications with renal-protective effects, such as sodium-glucose cotransporter 2 inhibitors, glucagon-like peptide-1 receptor agonists, and finerenone, are currently recommended for patients with DKDs [6]. Risk equation-based transition models, which comprehensively simulate the disease progression of DKDs, are useful for determining the specific renal effect of these medications regarding different DKD stages or progression and guiding personalized medicine. From a research perspective, this study offers a roadmap for model validation and recalibration. Notably, several risk prediction models for renal events are available today, but few have been externally validated and continuously updated, thereby restricting their generalizability to different settings and modern practice. Instead of developing a risk equation or model from scratch, adapting available risk equations and models to target populations or settings is generally suggested as an efficient approach [14, 24]. A risk equation with risk predictors that are transparently reported and routinely collected in clinical practice is usually preferred for further adaptation (e.g., external validation, updating, recalibration) [25].

Limitations

First, several novel biomarkers (e.g., vascular adhesion protein-1, inflammatory chemokine CXCL12) [26, 27] that may be associated with renal progression were not considered in our risk equations. However, these biomarkers are not routinely measured in clinical settings and thus may have limited real-world applicability. Risk equations that contain risk factors or predictors that are routinely collected were thus selected in this study to enhance the applicability of our work to routine clinical practice. Second, the discrimination of the recalibrated RECODe risk equation for microalbuminuria remains unsatisfactory. A low discrimination of the RECODe model for microalbuminuria had been found in its original development cohort (AUROC: 0.62) [7]. Therefore, future research that will utilize more advanced methodologies or develop an updated risk equation is warranted to improve the predictive performance for microalbuminuria. It should be noted, however, that for a risk equation that aims to accurately predict risks of patients, its performance of discrimination might be less relevant than that of calibration [14]. Third, the information bias due to loss to follow-up, a common issue inherent to studies using EHRs, cannot be fully ruled out in this study. Several efforts were made to mitigate this concern, including restricting the study patients to those having at least two diagnoses of T2D for entry into the study cohort and at least one record of eGFR and UACR in both the baseline and follow-up period. Forth, the presence of a renal event of interest was not determined using a renal biopsy, since such a procedure is expensive and not routinely performed in clinical practice in Taiwan. Instead, we applied two consecutive laboratory records (e.g., eGFR less than 15 mL/min/1.73m2) separated by at least 90 days to confirm a renal event (e.g., microalbuminuria) in the primary analyses, and further conducted the sensitivity analyses which used the restricted or refined definitions of renal events (e.g., as the presence of at least one eGFR value less than 15 mL/min/1.73 m2 and at least one ICD diagnosis code; S5 Table). Consistent results between the primary and sensitivity analyses might have supported the robustness of our findings. Lastly, due to the unavailability of diabetes duration in NCKUH, we assumed a 5-year duration for all patients in this study, which might potentially diminish discriminative ability of our risk equations. However, the discriminative performance of our risk equations was consistent with that of the original risk equations, suggesting that such assumption might not affect the performance of the risk equations among Taiwanese T2D populations. Additionally, other risk predictors such as the history of diabetes-related complications, HbA1c data and use of glucose-lowering agents may have served as proxy indicators for diabetes duration in the risk equations.