Background Prognostic models are becoming increasingly relevant in clinical trials as potential surrogate endpoints, and for patient management as clinical decision support tools. However, the impact of competing risks on model performance remains poorly investigated. We aimed to carefully assess the performance of competing risks and non-competing risks models in the context of kidney transplantation, where allograft failure and death with a functioning graft are two competing outcomes.

Methods We included 10 546 adult kidney transplant recipients enrolled in 10 countries (3941 patients in the derivation cohort, 6605 patients in international external validation cohorts). We developed prediction models for long-term kidney graft failure prediction, without accounting (i.e., censoring) and accounting for the competing risk of death with a functioning graft, using Cox and Fine-Gray regression models. To this aim, we followed a detailed and transparent analytical framework for competing and non-competing risks modelling, and carefully assessed the models’ development, stability, discrimination, calibration, overall fit, and generalizability in external validation cohorts and subpopulations. In total, 15 metrics were used to provide an exhaustive assessment of model performance.

Results Among the 3941 recipients included in the derivation cohort, 538 (13.65%) lost their graft and 414 (10.50%) died after a median follow-up post-risk evaluation of 5.77 years (IQR 3.52-7.00). In the external validation cohorts, 896 (13.56%) graft losses and 525 (7.95%) deaths occurred after a median follow-up post-risk evaluation of 4.25 years (IQR 2.35-6.59). At 7 years post-risk evaluation, overestimation of the cumulative incidence was moderate when using Kaplan-Meier, compared to the Aalen-Johansen estimate (16.71% versus 15.67% in the derivation cohort). Cox and Fine-Gray models for predicting the long-term graft failure exhibited similar and stable risk estimates (average MAPE of 0.0140 and 0.0138 for Cox and Fine-Gray models, respectively). At 7 years post-risk evaluation, discrimination and overall fit were good and comparable in the external validation cohorts (concordance index ranging from 0.76 to 0.86, Brier Scores ranging from 0.102 to 0.141). In a large series of subpopulations and clinical scenarios, both models performed well and similarly.

Conclusions Competing and non-competing risks models performed similarly in predicting long-term kidney graft failure. These results should be interpreted in light of the low rate of the competing event in our cohort, and do not stand as a general conclusion for competing risks modelling. Depending on the clinical scenario and the population considered, competing risks may be crucial to consider for accurate risk predictions.

The authors have declared no competing interest.

INSERM-Action thematique incitative sur programme Avenir (ATIP-Avenir) provided financial support; OA received a grant from the Fondation Bettencourt Schueller; academic grant support was provided by the non-profit organizations MSD Avenir and OrganX.

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The institutional review boards with oversight for patients at each participating center gave ethical approval for this work. Each patient from the Paris Transplant Group cohort provided written informed consent to be included in the Paris Transplant Group database. This database has been approved by the National French Commission for Bioinformatics, Data, and Patient Liberty: CNIL registration number: 363505. The Institutional Review Boards of Necker and Saint-Louis Hospitals approved the Paris Transplant Group's cohort. For the validation sets, data were collected as part of routine clinical practice and entered in centers' databases in compliance with local and national regulatory requirements and sent anonymized to the Paris Transplant Group. These validation cohorts followed the rules applied in each country. As part of research collaborations, the institutional review boards with oversight for patients at each center agreed to send the anonymized data to the Paris Transplant Group. A data audit was then conducted to ensure that the data were sufficient quality for analysis. In each cohort, patients gave written informed consent at the day of transplantation and were over 18 years of age.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

All data produced in the present study are available upon reasonable request to the authors