What is the status of the following scores and appropriate code as of April 2017?
(taken from https://github.com/MIT-LCP/mimic-code/tree/master/concepts/severityscores)
Beta? Production? Good to use?
I would like to use code for SOFA/SAPS/OASIS but I have no idea how trustworthy these codes are.
The Oxford Acute Severity of Illness Score (OASIS) is a parsimonious severity score developed using a hybrid genetic algorithm and particle swarm optimization approach which allowed direct optimization of a severity score in a clinically relevant form with simultaneous multivariate feature selection (Johnson, 2013). The new score discriminated better than the Acute Physiology Score III (APS III) when evaluated in a univariate fashion and equivalently when evaluated as a covariate in a larger risk adjustment model similar to APACHE IV. OASIS was designed to have an extremely low burden for data collection and quality control, requiring only 10 features, and not requiring laboratory measurements, diagnosis or comorbidity information.
The Simplified Acute Physiology Score (SAPS) was intended as a simplification of the Acute Physiology Score (APS), reducing the number of physiological parameters required from the original 34 to 13 plus age (LeGall, 1984). The variables chosen were present for 90% of patients in the initial survey used to develop the APS (Knaus, 1981). Though the original publication only provided ROC curves and not the AUROC, trapezoidal integration showed that SAPS had an AUROC of 0.7697 and APS had an AUROC of 0.7661.
The Sepsis-related Organ Failure Assessment score was first developed by a consensus meeting of the ESICM in October 1994, though it eventually became known as the Sequential Organ Failure Assessment (SOFA) score as it was applied outside of septic populations (Vincent, 1996). The purpose of the score was to provide the clinical community with an objective measure of the severity of organ dysfunction in a patient. It is stressed that the score is not meant as a direct predictor of mortality but rather a measure of morbidity, or the level of the diseased state, in a patient. The score is evaluated for 6 organ systems: pulmonary, renal, hepatic, cardiovascular, haematologic and neurologic. Note that the organ-specific morbidity scores are highlighted as a useful compliment to the overall score.
The following severity scores have yet to be implemented, but are desirable, and contributions are welcome.
The Simplified Acute Physiology Score II (SAPS II) published in 1993 (Le Gall, 1993), aimed to rectify two issues with SAPS. First, the variable selection process in SAPS was done by clinical judgement, whereas SAPS II utilised univariate feature selection to filter out features uncorrelated with hospital mortality. Second, there was no model for calculating a probability of mortality from SAPS. SAPS II was published with calibration coefficients which allowed conversion of the integer score into a risk of mortality which ranged between zero and one.
The Simplified Acute Physiology Score III (SAPS III) is the latest model published and attempts to account for poor calibration of SAPS II found in later studies (Metnitz, 2005). While SAPS II was developed on ICUs in western Europe, SAPS III included ICUs worldwide. The parameters of the model were identified using a stepwise logistic regression, followed by statistical hypothesis testing to ensure the parameters were significantly related to patient hospital mortality (Moreno, 2005). The final prediction used 61 binary indicator features which dichotomised various ranges of 20 distinct variables (e.g. the highest heart rate variable was split into three binary features, one for heart rates $= 160). A hierarchical model was used, specifying patient characteristics as fixed effects and different ICUs as a random effect. A logistic regression equation was then used to calculate the probability of mortality.
APACHE IV is the most recent iteration in the Acute Physiology, Age, and Chronic Health Evaluation system. The model was published in 2006 and coefficients are freely available online. Overall, APACHE IV tends to have the best discriminative performance among the most recent generation of general purpose severity of illness models.
APACHE IV's components include: the Acute Physiology Score (APS) III, comorbidities, diagnosis, admission source, surgical status, mechanical ventilation flags and existence of thrombolytic therapy. Unfortunately, the diagnostic component of APACHE IV is very difficult to code.