External validation of two clinical prediction models for mortality in COVID-19 patients (4C and NEWS2), in three centers in Medellín, Colombia: Assessing the impact of vaccination over time

Abstract

ABSTRACT: Objectives: External validation of the 4C and NEWS2 scores for the prediction of in-hospital mortality in COVID-19 patients, and evaluation of its operational performance in two time periods: before and after the start of the vaccination program in Colombia. Methods: Retrospective cohort in three high complexity hospitals in the city of Medellín, Colombia, between June 2020 and April 2022. Results: The areas under the ROC curve (AUC) for the 4C mortality risk score and the NEWS2 were 0.75 (95% CI 0.73-0.78) and 0.68 (95% CI 0.66-0.71), respectively. For the 4C score, the AUC for the first and second periods was 0.77 (95% CI 0.74-0.80) and 0.75 (95% CI 0.71-0.78); whilst for the NEWS2 score, it was 0.68 (95% CI 0.65-0.71) and 0.69 (95% CI 0.64-0.73). The calibration for both scores was adequate, albeit with reduced performance during the second period. Conclusions: The 4C mortality risk score proved to be the more adequate predictor of in-hospital mortality in COVID-19 patients in this Latin American population. The operational performance during both time periods remained similar, which shows its utility notwithstanding major changes, including vaccination, as the pandemic evolved. Keywords: 4C Mortality Score; COVID-19; Mortality; The National Early Warning Score 2 (NEWS2); Validation studies.