SARS-CoV-2, the virus that causes COVID-19, has infected more than 103 million people worldwide. Acute kidney injury (AKI) treated with dialysis was a common complication in patients who were hospitalized with COVID-19. Acute kidney injury is associated with increased risks for morbidity and mortality. Early prediction of which patients will need dialysis or experience critical illness leading to mortality during hospital care can enhance appropriate monitoring, and better inform conversations with patients and their caretakers.
Results: The Mount Sinai team developed and tested five different algorithms to predict patients requiring treatment with dialysis or critical illness leading to death on day 1, 3, 5, and 7 of the hospital stay, using data from the first 12 hours of admission to the Mount Sinai Health System. Out of the five models, the XGBoost without imputation method, outperformed all others with higher precision and recall.
Why the Research Is Interesting: While the Mount Sinai model requires further external review, such machine learning models can potentially be deployed throughout healthcare systems to help determine which COVID-19 patients are most at risk for adverse outcomes of the coronavirus. Early recognition of at-risk patients can enhance closer monitoring of patients and prompt earlier discussions regarding goals of care.
Who: More than 6,000 adults with COVID-19 admitted to five hospitals within the Mount Sinai Health System.
When: COVID-19 patients admitted from March 10 to December 26, 2020.
What: The study uses a machine learning model to determine COVID-19 patients most at risk for treatment requiring dialysis or critical illness leading to death.
How: The team used data on adults hospitalized with COVID-19 throughout the Mount Sinai Health System to develop and validate five models for predicting treatment with dialysis or death at various time periods –1, 3, 5 and 7 days –following hospital admission. Patients admitted to Mount Sinai Hospital in Manhattan were used for internal validation, while the other four hospital locations were part of the external validation cohort. Assessed features included demographics, comorbidities, laboratory results, and vital signs within 12 hour of hospital admission.
The five models created and tested were: the logistic regression, LASSO, random forest, and XGBoost with and without imputation. Out of the total model approaches used, XGBoost without imputation had the highest area under the receiver curve and area under the precision recall curve on internal validation for all time points. This model also had the highest test parameters on external validation across all time windows. Features including red cell distribution width, creatinine, and blood urea nitrogen were major drivers of model prediction.
Study Conclusions: Mount Sinai researchers have developed and validated a machine learning model to identify hospitalized COVID-19 patients at risk of acute kidney injury and death. The XGBoost model without imputation had the best performance compared to standard and other machine learning models. Widespread use of electronic health records makes the deployment of prediction models, such as this one, possible.
Said Mount Sinai’s Dr. Girish Nadkarni of the research: The near universal use of electronic health records has created a tremendous amount of data, which has enabled us to generate prediction models that can directly aid in the care of patients. A version of this model is currently deployed at Mount Sinai Hospital in patients who are admitted with COVID-19.
Said Mount Sinai’s Dr. Lili Chan of the research: As a nephrologist, we were overwhelmed with the increase in patients who had AKI during the initial surge of the COVID-19 pandemic. Prediction models like this enable us to identify, early on in the hospital course, those at risk of severe AKI (those that required dialysis) and death. This information will facilitate clinical care of patients and inform discussions with patients and their families.
Said Mount Sinai’s Dr. Akhil Vaid of the research: Machine learning allows us to discern complex patterns in large amounts of data. For COVID-19 inpatients, this means being able to more easily identify incoming at-risk patients, while pinpointing the underlying factors that are making them better or worse. The underlying algorithm, XGBoost, excels in accuracy, speed, and other under-the-hood features that allow for easier deployment and understanding of model predictions.
