Motivation Acute kidney injury (AKI) is a syndrome that affects a large fraction of all critically ill patients, and early diagnosis to receive adequate treatment is as imperative as it is challenging to make early. Consequently, machine learning approaches have been developed to predict AKI ahead of time. However, the prevalence of AKI is often underestimated in state-of-the-art approaches, as they rely on an AKI event annotation solely based on creatinine, ignoring urine output.

Methods We construct and evaluate early warning systems for AKI in a multi-disciplinary ICU setting, using the complete KDIGO definition of AKI. We propose several variants of gradient-boosted decision trees (GBDT)-based models, including a novel time-stacking based approach. A state-of-the-art LSTM-based model previously proposed for AKI prediction is used as a comparison, which was not specifically evaluated in ICU settings yet.

Results We find that optimal performance is achieved by using GBDT with the time-based stacking technique (AUPRC=65.7%, compared with the LSTM-based model’s AUPRC=62.6%), which is motivated by the high relevance of time since ICU admission for this task. Both models show mildly reduced performance in the limited training data setting, perform fairly across different subco-horts, and exhibit no issues in gender transfer.

Conclusion Following the official KDIGO definition substantially increases the number of annotated AKI events. In our study GBDTs outperform LSTM models for AKI prediction. Generally, we find that both model types are robust in a variety of challenging settings arising for ICU data.

Competing Interests Statement: G.R. and K.B. are cofounders and members of the scientific board of Computomics GmbH.

Funding statement This project was supported by the Grant No. 205321_176005 of the Swiss National Science Foundation (to T.M.M./G.R.), and grant #2022-278 of the Strategic Focus Area for Personalized Health and Related Technologies (PHRT) of the ETH Domain (Swiss Federal Institutes of Technology), and ETH core funding (to G.R.), and the Horizon 2020 research and innovation programme of the European Union under the Marie Sklodowska-Curie grant agreement No 813533 (to K.B.).

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

Approval from ethics committee The institutional review board (IRB) of the Canton of Bern approved the study (BASEC 2016 01463). The need for obtaining informed patient consent for patient data from our institution was waived owing to the retrospective and observational nature of the study. No IRB approval is required for the anonymous public external validation data-set from Amsterdam University Medical Center.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Data availability statement: Data used in this study were obtained from University Hospital Bern and we cite Hüser et al. for its use (https://www.medrxiv.org/content/10.1101/2024.01.23.24301516v1).